CLI Reference¶

Command and option tables are generated from the Click CLI at build time; the section notes below are hand-written. Run hawk --help locally for the same detail. When a command accepts an optional [EVAL_SET_ID] / [JOB_ID], omitting it uses the last eval set from the current session.

Authentication¶

Log in once with hawk login; the token is cached and refreshed automatically. On headless hosts (containers, SSH) use --no-browser, which takes the device code flow where the provider supports it and otherwise prints a URL you can open on any device and paste the redirect back. --device forces the device flow, which the default Cognito auth does not support. The auth subcommands print raw tokens for scripting; they never start an interactive login and instead exit with an error when you are not logged in or the session has expired.

Log in to the Hawk API. Opens a browser for OAuth2 login (PKCE); falls back to a paste-back flow when no browser is available.

Options:

Name	Type	Description	Default
`--device`	boolean	Force device code flow (requires an OIDC provider with device-flow support; AWS Cognito does not have it)	`False`
`--no-browser`	boolean	Don't open a browser (headless/SSH environments): use the device code flow if the OIDC provider supports it, otherwise print the login URL to open on any device and paste back the redirect URL	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk auth¶

Authentication-related commands.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Subcommands

access-token:
login:
refresh-token:

hawk auth access-token¶

Print a valid access token to stdout.

Refreshes an expired access token when possible. Intended for scripting, so it never starts an interactive login: exits with an error if you are not logged in or the session has expired.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Log in to the Hawk API. Opens a browser for OAuth2 login (PKCE); falls back to a paste-back flow when no browser is available.

Options:

Name	Type	Description	Default
`--device`	boolean	Force device code flow (requires an OIDC provider with device-flow support; AWS Cognito does not have it)	`False`
`--no-browser`	boolean	Don't open a browser (headless/SSH environments): use the device code flow if the OIDC provider supports it, otherwise print the login URL to open on any device and paste back the redirect URL	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk auth refresh-token¶

Print the current refresh token.

Never starts an interactive login: exits with an error if no refresh token is stored.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Evaluations¶

hawk eval-set CONFIG submits an eval set (a YAML config — see Running Evaluations and the Eval Set Config Reference). hawk eval-set resume restarts a crashed run from its last checkpoint.

resume with --config FILE lets you resume with an updated, checkpoint-compatible config (e.g. to fix a crashing scorer); it requires an explicit ID and does not validate compatibility — see Checkpointing & Resume for what's safe to change. If the previous run has finished (or crashed), resume clears its leftover release automatically. It refuses (409) only when that run is still running — stop it first (hawk stop or hawk delete) — or when its state can't be confirmed, in which case clear it with hawk delete and retry.

Sample-level checkpointing is configured in the eval-set YAML (checkpoint.enabled); see Checkpointing & Resume for requirements, agent support, and the resume workflow.

hawk eval-set¶

Run, resume, and manage Inspect eval sets.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Subcommands

resume: Resume a previously-launched eval set from its last checkpoint.
run: Run an Inspect eval set remotely.

hawk eval-set resume¶

Resume a previously-launched eval set from its last checkpoint.

By default the eval-set configuration is restored from the state saved when the eval set was first created; pass --config to relaunch with an updated, checkpoint-compatible config (e.g. to fix a crashing scorer). Secrets must be re-provided via --secret/--secrets-file. Only checkpointed samples resume in progress; others restart from the beginning.

A leftover runner release from the finished/crashed run is cleared automatically (resume reads the Kubernetes Job status and only tears down a provably-finished Job). Resume fails with a conflict only if that run is still running -- stop it first with hawk stop or hawk delete -- or if its state can't be confirmed, in which case run hawk delete <id> and retry.

If no EVAL_SET_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-eval-set-id).

Options:

Name	Type	Description	Default
`--image`	text	Full container image URI for the runner (e.g., 'ghcr.io/org/runner:v1')	None
`--image-tag`	text	Inspect image tag	None
`--secrets-file`	file	Secrets file to load environment variables from	None
`--secret`	text	Name of environment variable to pass as secret (can be used multiple times)	None
`--config`	file	Resume with this updated eval-set config (e.g. to fix a crashing scorer or bump a task pin). Must be checkpoint-compatible. Requires an explicit EVAL_SET_ID.	None
`--skip-confirm`	boolean	Skip the unknown-field confirmation when validating --config	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk eval-set run¶

Run an Inspect eval set remotely.

EVAL_SET_CONFIG_FILE is a YAML file that contains a grid of tasks, solvers, and models. This configuration will be passed to the Inspect API and then an Inspect "runner" job, where the eval set will be run.

You can set environment variables for the environment where the Inspect process will run using --secret or --secrets-file. These work for non-sensitive environment variables as well, not just "secrets", but they're all treated as sensitive just in case.

By default, API calls to model providers detected in your eval-set configuration are automatically redirected to an LLM proxy server and use OAuth JWTs (instead of real API keys) for authentication. This includes native providers (OpenAI, Anthropic, Google Vertex) as well as OpenAI-compatible providers accessed via the openai-api/<provider>/<model> pattern (e.g., OpenRouter, DeepSeek, Groq, Together, and others).

The following environment variables are automatically set for convinience: - BASE_API_KEY: API key for the LLM proxy (your OAuth JWT) - AI_GATEWAY_BASE_URL: Base URL of the LLM proxy server

As an escape hatch (e.g. in case our LLM proxy server doesn't support some newly released feature or model), you can override provider API keys and base URLs using --secret. NOTE: you should only use this as a last resort, and this functionality might be removed in the future.

Options:

Name	Type	Description	Default
`--image`	text	Full container image URI for the runner (e.g., 'ghcr.io/org/runner:v1')	None
`--image-tag`	text	Inspect image tag	None
`--secrets-file`	file	Secrets file to load environment variables from	None
`--secret`	text	Name of environment variable to pass as secret (can be used multiple times)	None
`--skip-confirm`	boolean	Skip confirmation prompt for unknown configuration warnings	`False`
`--log-dir-allow-dirty`	boolean	Allow unrelated eval logs to be present in the log directory	`False`
`--monitor`	boolean	Enable Datadog monitoring. Alerts via Slack DM if errors appear during execution.	`False`
`--help`	boolean	Show this message and exit.	`False`

Scans¶

hawk scan run CONFIG starts a Scout scan; hawk scan resume continues an interrupted one. hawk scan import PATH ingests a locally-produced scan directory (or a folder of them) into the warehouse under a single run — pass --name NAME to set a friendly slug. The scanned transcripts must already be in the warehouse.

hawk scan¶

Run and manage Scout scans.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Subcommands

import: Upload locally-produced Scout scan(s) into Hawk's warehouse.
resume: Resume a Scout scan.
run: Run a Scout Scan remotely.

hawk scan import¶

Upload locally-produced Scout scan(s) into Hawk's warehouse.

PATH is either a single Scout scan results directory (the scan_id=... directory Scout writes) or a folder of such directories. Each scan's _scan.json, _summary.json, and per-scanner .parquet files are uploaded under one newly-allocated, imported--prefixed scan run.

Options:

Name	Type	Description	Default
`--name`	text	Friendly name for the imported scan (used in the auto-generated scan_run_id).	None
`--help`	boolean	Show this message and exit.	`False`

hawk scan resume¶

Resume a Scout scan.

The scan configuration is restored from the state saved when the scan was first created. Secrets must be re-provided via --secret or --secrets-file.

If no SCAN_RUN_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-scan-run-id).

Options:

Name	Type	Description	Default
`--image`	text	Full container image URI for the runner (e.g., 'ghcr.io/org/runner:v1')	None
`--image-tag`	text	Inspect image tag	None
`--secrets-file`	file	Secrets file to load environment variables from	None
`--secret`	text	Name of environment variable to pass as secret (can be used multiple times)	None
`--allow-sensitive-cross-lab-scan`	boolean	Allow scanning private model transcripts with scanners from a different lab.	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk scan run¶

Run a Scout Scan remotely.

SCAN_CONFIG_FILE is a YAML file that contains a matrix of scanners and models. This configuration will be passed to the Inspect API and then an Inspect "runner" job, where the scan will be run.

You can set environment variables for the environment where the Inspect process will run using --secret or --secrets-file. These work for non-sensitive environment variables as well, not just "secrets", but they're all treated as sensitive just in case.

By default, API calls to model providers detected in your scan configuration are automatically redirected to an LLM proxy server and use OAuth JWTs (instead of real API keys) for authentication. This includes native providers (OpenAI, Anthropic, Google Vertex) as well as OpenAI-compatible providers accessed via the openai-api/<provider>/<model> pattern (e.g., OpenRouter, DeepSeek, Groq, Together, and others).

The following environment variables are automatically set for convinience: - BASE_API_KEY: API key for the LLM proxy (your OAuth JWT) - AI_GATEWAY_BASE_URL: Base URL of the LLM proxy server

As an escape hatch (e.g. in case our LLM proxy server doesn't support some newly released feature or model), you can override provider API keys and base URLs using --secret. NOTE: you should only use this as a last resort, and this functionality might be removed in the future.

Options:

Name	Type	Description	Default
`--image`	text	Full container image URI for the runner (e.g., 'ghcr.io/org/runner:v1')	None
`--image-tag`	text	Inspect image tag	None
`--secrets-file`	file	Secrets file to load environment variables from	None
`--secret`	text	Name of environment variable to pass as secret (can be used multiple times)	None
`--skip-confirm`	boolean	Skip confirmation prompt for unknown configuration warnings	`False`
`--allow-sensitive-cross-lab-scan`	boolean	Allow scanning private model transcripts with scanners from a different lab.	`False`
`--help`	boolean	Show this message and exit.	`False`

Running locally¶

hawk local eval-set and hawk local scan run a config on your own machine instead of submitting it to the cluster — useful for quick iteration.

hawk local¶

Run evaluations and scans locally.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Subcommands

eval-set: Run an Inspect eval set locally.
scan: Run a Scout scan locally.

hawk local eval-set¶

Run an Inspect eval set locally.

Requires installing hawk with the runner extra.

CONFIG_FILE is a YAML file with the eval set configuration.

Options:

Name	Type	Description	Default
`--direct`	boolean	Run in current environment instead of creating a new venv	`False`
`--secrets-file`	file	Secrets file to load environment variables from	None
`--secret`	text	Name of environment variable to pass as secret (can be used multiple times)	None
`--help`	boolean	Show this message and exit.	`False`

hawk local scan¶

Run a Scout scan locally.

Requires installing hawk with the runner extra.

CONFIG_FILE is a YAML file with the scan configuration.

Options:

Name	Type	Description	Default
`--direct`	boolean	Run in current environment instead of creating a new venv	`False`
`--secrets-file`	file	Secrets file to load environment variables from	None
`--secret`	text	Name of environment variable to pass as secret (can be used multiple times)	None
`--help`	boolean	Show this message and exit.	`False`

Monitoring¶

Watch a running job with hawk watch (live per-task / per-sample view — the same one the web viewer shows, streamed over SSE with a polling fallback). hawk logs tails the job's logs, hawk status prints a one-shot JSON report.

hawk trace streams the Inspect trace log — enter/exit events for model calls, subprocesses, and other long-running actions — which is the fastest way to see where a stuck eval is blocked. Records format like hawk logs output; --raw emits the underlying JSON-lines for inspect trace anomalies. It needs the same model-group permissions as viewing results and only works while the runner pod is live.

hawk status performance statistics cover imported samples only

The performance block is aggregated from the warehouse, and samples reach the warehouse only once their eval log is written to S3 and imported. Samples that are still running are not in the warehouse yet, so they contribute nothing to the sums or counts — a report taken mid-run describes the work finished so far, not the work in flight. On an eval set where nothing has been imported yet, the timing sums are null and the counts are 0.

For in-flight progress use hawk watch, which reads the runner's live sample buffer instead of the warehouse. Importing running samples into the warehouse is planned; until then the two commands answer different questions.

hawk trace abc123 -f                            # Follow new trace lines live (Ctrl-C to stop)
hawk trace abc123 --full --raw > trace.log      # Raw trace for: inspect trace anomalies trace.log

hawk stacktrace captures a live py-spy dump of the runner process (PID 1) — shows exactly where each thread is blocked right now. Live only.

hawk logs¶

View logs for a job.

Examples: hawk logs abc123 # Show last 100 logs hawk logs abc123 -n 50 # Show last 50 lines hawk logs -f # Follow mode (Ctrl+C to stop)

If no JOB_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-job-id).

Options:

Name	Type	Description	Default
`-n`, `--lines`	integer	Number of lines to show (default: 100)	`100`
`-f`, `--follow`	boolean	Follow mode - continuously poll for new logs	`False`
`--hours`	integer	Hours of data to search (default: 5 years)	`43800`
`--poll-interval`	float	Seconds between polls in follow mode (default: 3.0)	`3.0`
`--help`	boolean	Show this message and exit.	`False`

hawk status¶

Generate a monitoring report for a job.

Fetches logs, metrics, pod status, etc. and returns it as JSON.

If no JOB_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-job-id).

Options:

Name	Type	Description	Default
`--hours`	integer	Hours of log data to fetch (default: 24)	`24`
`--help`	boolean	Show this message and exit.	`False`

hawk watch¶

Watch live eval-set status: per-task / per-sample progress, retries, limits, and scheduling/pod trouble.

Streams updates via SSE until the run finishes (Ctrl-C to stop). Use --no-follow for a one-shot snapshot, or --json for the raw payload.

Options:

Name	Type	Description	Default
`--no-follow`	boolean	Print a single snapshot and exit instead of streaming live updates.	`False`
`--json`	boolean	Output the raw status JSON (implies a single snapshot).	`False`
`--help`	boolean	Show this message and exit.	`False`

Interactive sessions¶

Attach an interactive client to a running eval's pod through the per-env relay: the CLI opens a loopback TCP bridge and tunnels it to the pod over an authenticated WebSocket, reusing your hawk login token.

hawk acp [RUN_ID] is shorthand for hawk attach [RUN_ID] with no --port; both default to the last job used by hawk on this machine when RUN_ID is omitted. The relay URL is discovered from the server (GET /auth/config); set HAWK_RELAY_URL to override. The loopback port is unauthenticated for the session's lifetime — any local process can use it to reach the pod as you, the same trust model as kubectl port-forward.

hawk acp¶

Attach to a running eval's ACP server via the relay.

Bridges a local loopback port to the run's ACP (Agent Client Protocol) server through the per-env relay and launches inspect acp against it (unless --no-launch). This fronts Inspect's own inspect acp client; ACP is the Agent Client Protocol (https://agentclientprotocol.com).

If no RUN_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-job-id).

Options:

Name	Type	Description	Default
`--launch` / `--no-launch`	boolean	Launch `inspect acp` against the bridge, or just print the address.	`True`
`--local-port`	integer range (between `1` and `65535`)	Bind the local bridge to this loopback port instead of an ephemeral one (useful with --no-launch so scripts connect to a known address).	None
`--help`	boolean	Show this message and exit.	`False`

hawk attach¶

Bridge a local TCP port to a running eval's pod via the relay.

Without --port the relay resolves the run's ACP server and launches inspect acp against it. With --port it bridges an arbitrary container port and does not launch inspect acp (which only speaks ACP) unless you pass --launch. See hawk acp for the ACP shorthand.

If no RUN_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-job-id).

Options:

Name	Type	Description	Default
`--port`	integer range (between `1` and `65535`)	Target this container port instead of the run's ACP server.	None
`--launch` / `--no-launch`	boolean	Launch `inspect acp` against the bridge (default: on, unless --port is given).	None
`--local-port`	integer range (between `1` and `65535`)	Bind the local bridge to this loopback port instead of an ephemeral one (useful with --no-launch so scripts connect to a known address).	None
`--help`	boolean	Show this message and exit.	`False`

Viewing Results¶

hawk trace¶

View the Inspect trace log from a job's running runner pod.

The trace log records enter/exit events for model calls, subprocesses, and other long-running actions — useful for diagnosing a stuck or in-progress eval. Only available while the runner pod is running.

Trace records are formatted like hawk logs output; pass --raw for the underlying JSON-lines (e.g. to analyze with inspect trace anomalies).

Examples: hawk trace abc123 # Last 100 trace lines hawk trace abc123 -f # Follow new lines (Ctrl-C to stop) hawk trace abc123 --full --raw > trace.log # Full raw trace; then: # inspect trace anomalies trace.log

If no JOB_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-job-id).

Options:

Name	Type	Description	Default
`-n`, `--lines`	integer range (between `1` and `50000`)	Number of lines to show from the end of the trace (default: 100; ignored with --full)	None
`--full`	boolean	Fetch the entire trace file instead of just the tail	`False`
`-f`, `--follow`	boolean	Follow the trace, printing new lines as the eval appends them (Ctrl-C to stop)	`False`
`--poll-interval`	float range (`0.5` and above)	Seconds between polls in --follow mode (default: 3.0)	`3.0`
`--raw`	boolean	Output the unmodified JSON-lines trace records instead of formatted lines (use for `inspect trace anomalies` and other tooling)	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk stacktrace¶

Capture a live Python stack trace of a job's running runner process.

Runs py-spy dump against the inspect runner (PID 1) inside the runner pod and prints the result — useful for diagnosing a stuck or in-progress eval, since it shows exactly where each thread is blocked right now. Only available while the runner pod is running.

Examples: hawk stacktrace abc123 # Text dump of the runner hawk stacktrace abc123 --json # py-spy JSON (for tooling) hawk stacktrace abc123 --native # Include native stack frames

If no JOB_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-job-id).

Options:

Name	Type	Description	Default
`--native`	boolean	Include native (C-extension) stack frames.	`False`
`--json`	boolean	Output py-spy's JSON instead of the formatted text dump.	`False`
`--help`	boolean	Show this message and exit.	`False`

Viewing results¶

hawk web and hawk view-sample open results in the browser. hawk list ... enumerates jobs, eval sets, evals, and samples. hawk transcript / hawk transcripts download sample transcripts (markdown by default, --raw for JSON).

hawk download fetches raw .eval log files via presigned S3 URLs. hawk download-artifacts fetches sample artifact files, written as <output-dir>/<task>_<model>_<sample-id>_epoch<n>_<sample-uuid>/<artifact-path>. Missing metadata is omitted; at minimum the UUID remains. Default output dir: artifacts/<eval-set-id>; existing files are overwritten.

hawk web¶

Open the eval set log viewer in your web browser.

If no EVAL_SET_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-eval-set-id).

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

hawk view-sample¶

Open the sample log viewer in your web browser.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

hawk list¶

List evaluations or samples in an eval set.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Subcommands

eval-sets: List eval sets
evals:
jobs: List launched jobs
samples:

hawk list eval-sets¶

List eval sets

Options:

Name	Type	Description	Default
`--limit`	integer	Maximum number of eval sets to show	`10`
`--search`	text	Filter eval sets	None
`--help`	boolean	Show this message and exit.	`False`

hawk list evals¶

List all evaluations in an eval set.

Shows task name, model, status, and sample counts for each evaluation.

If no EVAL_SET_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-eval-set-id).

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

hawk list jobs¶

List launched jobs (eval-sets and scans).

Options:

Name	Type	Description	Default
`--all`	boolean	Show all jobs the caller can see (default: only the caller's own jobs)	`False`
`--limit`	integer	Maximum number of jobs to show	`10`
`--help`	boolean	Show this message and exit.	`False`

hawk list samples¶

List samples within an eval set.

Shows sample UUID, ID, epoch, status, and scores for each sample.

If no EVAL_SET_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-eval-set-id).

Options:

Name	Type	Description	Default
`--eval`	text	Filter to a specific eval file	None
`--limit`	integer	Maximum number of samples to show	`50`
`--json`	boolean	Output raw JSON instead of a table (includes each sample's scores, with intermediates flagged)	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk transcript¶

Download transcript for a single sample.

Shows all conversation turns with role, content, tool calls, and scores.

Options:

Name	Type	Description	Default
`--output-dir`	path	Write transcript to a file in this directory	None
`--raw`	boolean	Output raw sample JSON instead of markdown	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk transcripts¶

Download transcripts for all samples in an eval set.

Fetches all samples and outputs them with separator headers. Use --output-dir to write individual files instead of stdout. Use --limit to restrict the number of samples.

If no EVAL_SET_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-eval-set-id).

Options:

Name	Type	Description	Default
`--output-dir`	path	Write transcripts to individual files in this directory	None
`--limit`	integer	Limit number of samples	None
`--raw`	boolean	Output raw sample JSON instead of markdown	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk download¶

Download eval log files via presigned S3 URLs.

Downloads eval files directly from S3, bypassing the S3 Object Lambda Access Point. Use this for large eval files that time out via the OLAP.

Use --list to list available eval files without downloading.

If no EVAL_SET_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-eval-set-id).

Options:

Name	Type	Description	Default
`--eval`	text	Download a specific eval file (by filename)	None
`--output-dir`, `-o`	directory	Output directory (default: current directory)	`.`
`--list`	boolean	List eval files without downloading	`False`
`--jobs`, `-j`	integer range (`1` and above)	Number of concurrent downloads.	`16`
`--force`	boolean	Re-download files even if they already exist locally.	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk download-artifacts¶

Download sample artifact files for an eval set.

Downloads the artifacts tree for all samples in an eval set, or for one sample when --sample is provided. Each sample's files land in a folder named epoch, with missing metadata omitted. Existing files are overwritten.

If no EVAL_SET_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-eval-set-id).

Options:

Name	Type	Description	Default
`--sample`	text	Download artifacts for a single sample UUID.	None
`--output-dir`, `-o`	directory	Output directory (default: artifacts/).	None
`--help`	boolean	Show this message and exit.	`False`

Management¶

hawk stop ends a run gracefully, scoring partial work; hawk delete tears down its Kubernetes resources immediately (logs are kept either way) and accepts an eval-set ID or a scan-run ID. hawk edit-samples submits sample edits, and hawk import ingests locally-produced .eval files into the warehouse.

hawk stop¶

Gracefully stop an eval set or a specific sample.

Interrupts in-progress samples and runs scoring before writing .eval files. Use --error to mark samples as failed (retryable via eval_retry) instead.

Unlike 'delete', this preserves all agent progress and transcripts.

If no EVAL_SET_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-eval-set-id).

Options:

Name	Type	Description	Default
`--sample`	text	UUID of a specific sample to stop (others continue running).	None
`--error`	boolean	Mark stopped samples as errors (retryable) instead of scoring them.	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk delete¶

Delete an eval set or scan run. Cleans up all the job's resources, including sandbox environments. Does not delete the job's logs.

If no JOB_ID is given, defaults to the last one used by hawk on this machine (~/.config/hawk-cli/last-job-id).

Options:

Name	Type	Description	Default
`--yes`, `-y`	boolean	Skip confirmation. Requires an explicit JOB_ID.	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk edit-samples¶

Submit sample edits to the Hawk API.

EDITS_FILE is a JSON or JSONL file containing sample edits.

For JSON files, the format should be an array of edit objects:

[ { "sample_uuid": "...", "details": { "type": "score_edit", ..., } }, { "sample_uuid": "...", "details": { "type": "invalidate_sample", ..., } }, ... ]

For JSONL files, each line should be a single edit object:

{"sample_uuid": "...", "details": {"type": "score_edit", ...}} {"sample_uuid": "...", "details": {"type": "invalidate_sample", ...}}

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

hawk import¶

Upload .eval files into Hawk's warehouse.

PATH may be a single .eval file or a directory containing one or more .eval files (searched recursively). All files are uploaded under a single newly-allocated eval-set ID.

Options:

Name	Type	Description	Default
`--name`	text	Friendly name for the imported eval set.	None
`--help`	boolean	Show this message and exit.	`False`

Humans¶

Manage external participants (and their SSH public keys) with hawk human ..., and run evaluations where a registered human works inside the sandbox — instead of an LLM agent — with hawk human eval .... hawk human eval start accepts any eval-set YAML: the server swaps in the configured human-agent solver and clamps epochs=1 / limit=1. hawk human eval ssh-command prints a ready-to-run ssh -J command that hops through the shared jumphost to the sandbox pod.

hawk human¶

Manage registered humans and run human evaluations.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Subcommands

delete: Delete a human.
eval: Run human evaluations.
list: List all registered humans.
register: Register a new human.
update: Update a human's SSH key.

hawk human delete¶

Delete a human.

Options:

Name	Type	Description	Default
`--force`	boolean	Discard eval-set assignment history and remove the human even if assignments exist.	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk human eval¶

Run human evaluations.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Subcommands

ssh-command: Print a copy-paste-ready SSH command for a human-eval sandbox.
start: Start a human evaluation.

hawk human eval ssh-command¶

Print a copy-paste-ready SSH command for a human-eval sandbox.

EVAL_SET_ID is the eval-set returned by hawk human eval start. If omitted, defaults to the most recently started eval-set.

Options:

Name	Type	Description	Default
`--timeout`	float	Seconds to wait for the agent's SSH connection line	`600.0`
`--skip-verify`	boolean	Print the connect command without checking that it works.	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk human eval start¶

Start a human evaluation.

EVAL_SET_CONFIG_FILE is a standard eval-set YAML config file. The --human flag specifies which registered human to assign.

Options:

Name	Type	Description	Default
`--human`	text	Name of the registered human to assign	_required
`--secrets-file`	file	Secrets file to load environment variables from	None
`--secret`	text	Name of environment variable to pass as secret (can be used multiple times)	None
`--rewrite` / `--no-rewrite`	boolean	Rewrite the config to use the operator-configured default human agent and clamp epochs/limit to 1. Pass --no-rewrite to send the config through unchanged (the SSH public key is still injected).	`True`
`--image`	text	Full container image URI for the runner (e.g., 'ghcr.io/org/runner:v1')	None
`--image-tag`	text	Inspect image tag	None
`--help`	boolean	Show this message and exit.	`False`

hawk human list¶

List all registered humans.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

hawk human register¶

Register a new human.

Options:

Name	Type	Description	Default
`--name`	text	Human-readable name (e.g., jane)	_required
`--ssh-key`	text	SSH public key string	_required
`--help`	boolean	Show this message and exit.	`False`

hawk human update¶

Update a human's SSH key.

Options:

Name	Type	Description	Default
`--ssh-key`	text	New SSH public key string	_required
`--help`	boolean	Show this message and exit.	`False`

Proxy administration¶

hawk proxy ... manages the Middleman LLM proxy — configured models and provider keys. See Middleman for the admin workflow.

hawk proxy¶

Manage the Middleman LLM proxy (models, provider keys).

Visible to all logged-in users; the server enforces admin authorization (require_admin_user) on each endpoint.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Subcommands

models: Manage model configurations.
secrets: Manage provider API keys.

hawk proxy models¶

Manage model configurations.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Subcommands

activate: Activate a previously deactivated model.
add: Add a model configuration.
deactivate: Deactivate a model so new requests can't use it.
get: Show one configured model.
list: List configured models.
reload: Reload model configs into running proxy workers.
update: Update a model configuration.

hawk proxy models activate¶

Activate a previously deactivated model.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

hawk proxy models add¶

Add a model configuration.

Options:

Name	Type	Description	Default
`--group`	text	Model group for access control.	_required
`--config`	text	JSON string or @path/to/file.json	_required
`--help`	boolean	Show this message and exit.	`False`

hawk proxy models deactivate¶

Deactivate a model so new requests can't use it.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

hawk proxy models get¶

Show one configured model.

Options:

Name	Type	Description	Default
`--format`	choice (`json` \| `table`)	Output format.	`json`
`--help`	boolean	Show this message and exit.	`False`

hawk proxy models list¶

List configured models.

Options:

Name	Type	Description	Default
`--status`	choice (`active` \| `inactive` \| `all`)	Filter by active/inactive status.	`active`
`--format`	choice (`json` \| `table`)	Output format.	`json`
`--help`	boolean	Show this message and exit.	`False`

hawk proxy models reload¶

Reload model configs into running proxy workers.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

hawk proxy models update¶

Update a model configuration.

Options:

Name	Type	Description	Default
`--config`	text	JSON string or @path/to/file.json	_required
`--help`	boolean	Show this message and exit.	`False`

hawk proxy secrets¶

Manage provider API keys.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Subcommands

delete: Delete a provider API key.
list: List providers that have an API key configured.
set: Set a provider key. Reads the value from stdin when piped, else prompts.

hawk proxy secrets delete¶

Delete a provider API key.

Options:

Name	Type	Description	Default
`--yes`, `-y`	boolean	Skip confirmation.	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk proxy secrets list¶

List providers that have an API key configured.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

hawk proxy secrets set¶

Set a provider key. Reads the value from stdin when piped, else prompts.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

Other¶

hawk config prints the current CLI configuration, hawk version compares the local and deployed versions, hawk models lists models reachable via the proxy, hawk usage reports recent spend, and hawk scan-export writes scan results to CSV.

hawk config¶

Print the current CLI configuration.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

hawk version¶

Show the hawk CLI version and, if reachable, the deployed server version.

Options:

Name	Type	Description	Default
`--help`	boolean	Show this message and exit.	`False`

hawk models¶

List models you have access to via the LLM proxy.

Examples: hawk models -i claude # All Claude models hawk models -i claude -i sonnet # Claude Sonnet models hawk models -i claude -x openrouter # Claude, excluding OpenRouter hawk models -v -i gpt-4o # Detailed info for GPT-4o models

Options:

Name	Type	Description	Default
`-i`, `--include`	text	Regex pattern to include (all must match). Can be repeated.	None
`-x`, `--exclude`	text	Regex pattern to exclude (any match excludes). Can be repeated.	None
`-v`, `--verbose`	boolean	Show full model info as JSON lines instead of just names.	`False`
`--include-dead`	boolean	Also list dead models, kept only for permission checks on old data. Requests to them fail.	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk usage¶

Show your own token usage per provider/model via the LLM proxy.

Examples: hawk usage # your tokens per model over the last 24h hawk usage --hours 168 # last 7 days hawk usage --live # the recent ~5-minute window hawk usage --json

Options:

Name	Type	Description	Default
`--hours`	integer	Lookback window in hours for the usage sum (default: 24).	`24`
`--live`	boolean	Show the recent ~5-minute live window instead of a lookback sum.	`False`
`--json`	boolean	Output raw JSON instead of a table.	`False`
`--help`	boolean	Show this message and exit.	`False`

hawk scan-export¶

Export scan results as CSV.

Note: The input and scan_events columns are excluded from the export to reduce file size and memory usage.

Options:

Name	Type	Description	Default
`--output`, `-o`	path	Output file path (defaults to current directory with auto-generated filename)	None
`--help`	boolean	Show this message and exit.	`False`

CLI Reference¶

Authentication¶

hawk login¶

hawk auth¶

hawk auth access-token¶

hawk auth login¶

hawk auth refresh-token¶

Evaluations¶

hawk eval-set¶

hawk eval-set resume¶

hawk eval-set run¶

Scans¶

hawk scan¶

hawk scan import¶

hawk scan resume¶

hawk scan run¶

Running locally¶

hawk local¶

hawk local eval-set¶

hawk local scan¶

Monitoring¶

hawk logs¶

hawk status¶

hawk watch¶

Interactive sessions¶

hawk acp¶

hawk attach¶

Viewing Results¶

hawk trace¶

hawk stacktrace¶

Viewing results¶

hawk web¶

hawk view-sample¶

hawk list¶

hawk list eval-sets¶

hawk list evals¶

hawk list jobs¶

hawk list samples¶

hawk transcript¶

hawk transcripts¶

hawk download¶

hawk download-artifacts¶

Management¶

hawk stop¶

hawk delete¶

hawk edit-samples¶

hawk import¶

Humans¶

hawk human¶

hawk human delete¶

hawk human eval¶

hawk human eval ssh-command¶

hawk human eval start¶

hawk human list¶

hawk human register¶

hawk human update¶

Proxy administration¶

hawk proxy¶

hawk proxy models¶

hawk proxy models activate¶

hawk proxy models add¶

hawk proxy models deactivate¶

hawk proxy models get¶

hawk proxy models list¶

hawk proxy models reload¶

hawk proxy models update¶

hawk proxy secrets¶

hawk proxy secrets delete¶

hawk proxy secrets list¶

hawk proxy secrets set¶

Other¶

hawk config¶

hawk version¶

hawk models¶

hawk usage¶

hawk scan-export¶