Installing the CLI¶

Prerequisites¶

Python 3.13+
uv for Python package management
Access to a Hawk deployment (API server URL + OAuth2 credentials)

(Node.js 22, pnpm, and Graphviz are only needed for developing Hawk itself, not for using the CLI — see Contributing.)

Install the CLI¶

Install from PyPI with uv (or pip):

uv tool install "hawk[cli]"
# or: pip install "hawk[cli]"

To match a specific Hawk deployment, pin the release: uv tool install "hawk[cli]==2.0.0" (run hawk version, or check <API_URL>/version, to see which version the server runs).

PyPI installs pull upstream Inspect AI

The published package depends on the upstream inspect-ai and inspect-scout PyPI releases. METR-operated deployments pin lightly-patched forks via [tool.uv.sources], which don't ship in package metadata — so a hawk local run from a PyPI install builds its runner venv with upstream Inspect, not the fork. This is expected and fine for normal use; cluster runs (which use the deployment's runner image) are unaffected. To match a deployment's exact Inspect build, install from source instead (below).

Install from source (dev)

To hack on the CLI or run the latest unreleased main:

# latest main
uv tool install "hawk[cli] @ git+https://github.com/METR/hawk#subdirectory=hawk"

# editable clone
git clone https://github.com/METR/hawk.git
cd hawk/hawk
uv pip install -e ".[cli]"

Configuration¶

Set these environment variables before using the CLI, or put them in a .env file:

export HAWK_API_URL=https://api.hawk.example.com

Variable	Required	Description
`HAWK_API_URL`	Yes	URL of your Hawk API server
`HAWK_LOG_VIEWER_URL`	No	URL for the web log viewer (overrides what the API advertises)
`HAWK_MIDDLEMAN_URL`	No	URL of the Middleman LLM proxy, used by `hawk models` (overrides what the API advertises)
`HAWK_RELAY_URL`	No	URL of the relay used by `hawk acp` / `hawk attach` (overrides what the API advertises)
`HAWK_AI_GATEWAY_URL`	No	LLM gateway URL for `hawk local` runs (local-only; not advertised by the API)
`HAWK_ACCESS_TOKEN`	No	Bearer token to use instead of `hawk login` — for CI and other non-interactive use
`HAWK_DATADOG_EVAL_SET_DASHBOARD_URL`	No	Datadog dashboard URL for eval sets
`HAWK_DATADOG_SCAN_DASHBOARD_URL`	No	Datadog dashboard URL for scans
`HAWK_RUNNER_LOG_FORMAT`	No	Set to `json` for structured runner logs (useful with `hawk local`)
`HAWK_VERSION_CHECK`	No	What to do when the CLI and server versions differ: `prompt` (default — warn and ask to continue), `warn` (warn only), `block` (abort), or `off`

OIDC configuration is discovered automatically from the API server — no OIDC env vars are needed for the CLI.

Each variable resolves independently — pin all URLs to one deployment

The CLI reads each variable separately from the environment, the current directory's .env, and the global ~/.config/hawk-cli/env; an explicit URL in the global file overrides auto-discovery.

Symptom: with a stale global config, API calls hit the new deployment while hawk models/hawk web silently talk to an old one — typically a cryptic invalid_issuer 401.

Fix: always set HAWK_API_URL, HAWK_MIDDLEMAN_URL, and HAWK_LOG_VIEWER_URL together for the same deployment — scripts/dev/generate-env.py emits all three.

INSPECT_LOG_ROOT_DIR

This is an Inspect-level variable used by the runner inside the cluster, not the CLI. Set it on the cluster (via Pulumi) or in your local environment only if you're running hawk local and want to override where .eval files are written.

Version compatibility¶

Hawk uses SemVer (e.g. 2.0.0). Before each command the CLI probes the API's GET /version and warns when the two have drifted out of sync — the common cause of a command that works for one person but not another. A differing major version means a backwards-incompatible change and is flagged as likely incompatible; same-major drift is compatible per SemVer. The API and middleman each expose GET /version and an X-Hawk-Server-Version response header; the drift check uses the API's (the deployment's version).

hawk version    # prints the local CLI version and the deployed server version

To pin the CLI to a deployment, install the matching release (e.g. uv tool install "hawk[cli]==2.0.0"; see Install the CLI). HAWK_VERSION_CHECK controls the reaction to drift:

prompt (default) — warn and ask to continue. Non-interactively (CI, scripts) there's no prompt, so it warns and proceeds — same as warn.
warn — warn and proceed.
block — abort on any drift (including SemVer-compatible patch/minor). Recommended for CI when you need an exact match.
off — skip the check.

Authentication¶

hawk login

This authenticates via the browser-based PKCE flow (a localhost callback on port 18922). OIDC configuration is fetched from the API server and cached locally. Running hawk login also refreshes the cached OIDC configuration.

On headless machines (SSH boxes, containers), use hawk login --no-browser. You don't need to know which OIDC provider your deployment uses — the CLI picks the best flow it supports:

Provider implements the device grant (Okta, Auth0, most enterprise IdPs) → the OAuth2 Device Authorization flow. Prints a short user code and a URL; approve it in a browser on any device and the CLI picks up the result on its own. Nothing to copy back.
Provider doesn't (the default Cognito auth — user pools have no device-authorization endpoint) → paste-back PKCE. Prints a login URL to open on any device, then you paste the resulting http://localhost:18922/callback?... redirect URL (which fails to load in your browser — that's expected) back into the CLI.

In the paste-back case, if you SSH with -L 18922:localhost:18922 the login completes automatically without pasting. You can also feed the URL in from a file: hawk login --no-browser < url.txt.

hawk login --device forces the device flow rather than letting the CLI choose, and fails with "device flow not supported by the current OIDC provider" if the provider lacks it — useful when you want a script to assert the flow instead of silently falling back.

For fully non-interactive use (CI, service accounts), set HAWK_ACCESS_TOKEN to a valid bearer token — the CLI uses it directly and skips hawk login.

Run your first eval¶

The example configs live in the repo, so this needs the cloned repo (paths are relative to the repo root):

cd ..   # back to the repo root if you're still in hawk/ from the source install
hawk eval-set hawk/examples/simple.eval-set.yaml
hawk logs -f    # watch it run
hawk web        # open results in browser