Skip to content

Installing the CLI

Prerequisites

  • Python 3.13+
  • uv for Python package management
  • Node.js 22 and pnpm for frontend development
  • Graphviz for schema diagram rendering
  • Access to a Hawk deployment (API server URL + OAuth2 credentials)

Install the CLI

To install the CLI from GitHub, use the install command in the main README.

From source:

git clone https://github.com/METR/hawk.git
cd hawk/hawk
uv pip install -e ".[cli]"

Configuration

Set these environment variables before using the CLI, or put them in a .env file:

export HAWK_API_URL=https://api.hawk.example.com
Variable Required Description
HAWK_API_URL Yes URL of your Hawk API server
HAWK_LOG_VIEWER_URL No URL for the web log viewer (overrides what the API advertises)
HAWK_MIDDLEMAN_URL No URL of the Middleman LLM proxy, used by hawk models (overrides what the API advertises)
HAWK_AI_GATEWAY_URL No LLM gateway URL for hawk local runs (overrides what the API advertises)
HAWK_DATADOG_EVAL_SET_DASHBOARD_URL No Datadog dashboard URL for eval sets
HAWK_DATADOG_SCAN_DASHBOARD_URL No Datadog dashboard URL for scans
HAWK_RUNNER_LOG_FORMAT No Set to json for structured runner logs (useful with hawk local)
HAWK_VERSION_CHECK No What to do when the CLI and server versions differ: prompt (default — warn and ask to continue), warn (warn only), block (abort), or off

OIDC configuration is discovered automatically from the API server — no OIDC env vars are needed for the CLI.

Each variable resolves independently — pin all URLs to one deployment

The CLI reads each variable separately from the environment, the current directory's .env, and the global ~/.config/hawk-cli/env; an explicit URL in the global file overrides auto-discovery.

Symptom: with a stale global config, API calls hit the new deployment while hawk models/hawk web silently talk to an old one — typically a cryptic invalid_issuer 401.

Fix: always set HAWK_API_URL, HAWK_MIDDLEMAN_URL, and HAWK_LOG_VIEWER_URL together for the same deployment — scripts/dev/generate-env.py emits all three.

INSPECT_LOG_ROOT_DIR

This is an Inspect-level variable used by the runner inside the cluster, not the CLI. Set it on the cluster (via Pulumi) or in your local environment only if you're running hawk local and want to override where .eval files are written.

Version compatibility

Hawk uses SemVer (e.g. 2.0.0). Before each command the CLI probes the API's GET /version and warns when the two have drifted out of sync — the common cause of a command that works for one person but not another. A differing major version means a backwards-incompatible change and is flagged as likely incompatible; same-major drift is compatible per SemVer. The API and middleman each expose GET /version and an X-Hawk-Server-Version response header; the drift check uses the API's (the deployment's version).

hawk version    # prints the local CLI version and the deployed server version

To pin the CLI to a deployment, install from the matching tag (see the install command). HAWK_VERSION_CHECK controls the reaction to drift:

  • prompt (default) — warn and ask to continue. Non-interactively (CI, scripts) there's no prompt, so it warns and proceeds — same as warn.
  • warn — warn and proceed.
  • block — abort on any drift (including SemVer-compatible patch/minor). Recommended for CI when you need an exact match.
  • off — skip the check.

Authentication

hawk login

This authenticates via the browser-based PKCE flow (a localhost callback on port 18922). OIDC configuration is fetched from the API server and cached locally. Running hawk login also refreshes the cached OIDC configuration.

hawk login --device uses the OAuth2 Device Authorization flow instead — print a URL + code, complete in any browser — but only when the OIDC provider implements the device grant (Okta, Auth0, and most enterprise IdPs do). The default Cognito auth does not: Cognito user pools have no device-authorization endpoint, so --device fails with "device flow not supported by the current OIDC provider". Unsure which provider your deployment uses? Just run hawk login — the PKCE flow works with every provider; reach for --device only on a host without a browser.

Headless login with Cognito

A Cognito-backed deployment has no fully headless login. On a remote/headless host, forward the PKCE callback to a machine with a browser and complete the flow there:

ssh -L 18922:localhost:18922 <remote-host>
# then, in that SSH session:
hawk login   # complete the login in your LOCAL browser

The command must run to completion — the token is stored only after the callback fires.

Run your first eval

hawk eval-set examples/simple.eval-set.yaml
hawk logs -f    # watch it run
hawk web        # open results in browser