Skip to content

Quick Start

This gets you from zero to a working Hawk deployment on AWS. You'll need an AWS account and a domain name. You can use your existing OIDC identity provider for authentication, or a Cognito user pool by default.

1. Install prerequisites

bash brew install pulumi awscli uv python@3.13 jq

Or on Linux, install Pulumi, uv, the AWS CLI, Python 3.13+, and jq.

2. Clone the repo

bash git clone https://github.com/METR/hawk.git cd hawk

3. Set up Pulumi state backend

bash aws configure # or: aws sso login --profile your-profile

Create an S3 bucket and KMS key for Pulumi state:

bash aws s3 mb s3://my-org-hawk-pulumi-state # must be globally unique aws kms create-alias --alias-name alias/pulumi-secrets \ --target-key-id $(aws kms create-key --query KeyMetadata.KeyId --output text)

Log in to the S3 backend:

bash pulumi login s3://my-org-hawk-pulumi-state

4. Create and configure your stack

bash cd infra pulumi stack init my-org --secrets-provider="awskms://alias/pulumi-secrets" cp ../Pulumi.example.yaml ../Pulumi.my-org.yaml

Edit Pulumi.my-org.yaml with your values. At minimum, you need:

yaml config: aws:region: us-west-2 hawk:domain: hawk.example.com # domain you control — used for API and service routing hawk:publicDomain: example.com # parent domain for DNS zones and TLS certs hawk:primarySubnetCidr: "10.0.0.0/16" hawk:createPublicZone: "true"

That's enough to get started. The environment name defaults to your stack name. Hawk will create a Cognito user pool for authentication automatically.

If you already have an OIDC provider (Okta, Auth0, etc.), you can use it instead:

```yaml

Optional: use your own OIDC provider instead of Cognito

hawk:oidcClientId: "your-client-id" hawk:oidcAudience: "your-audience" hawk:oidcIssuer: "https://login.example.com/oauth2/default" ```

5. Deploy

bash pulumi up

This creates roughly 200+ AWS resources including a VPC, EKS cluster, ALB, ECS services, Aurora PostgreSQL, S3 buckets, Lambda functions, and more. First deploy takes about 15-20 minutes.

First deploy? TLS certificate validation requires working DNS. If this is a new hosted zone, the first deploy may fail with a certificate error — just run pulumi up again after a few minutes. If your domain is registered outside Route53, you'll also need to update your registrar's nameservers to match the new Route53 hosted zone before retrying. In the AWS console, go to Route53 → Hosted zones → your zone and copy the four NS record values. At your registrar, replace your domain's nameservers with those values.

6. Set up LLM API keys

Hawk routes model API calls through its built-in LLM proxy (Middleman). You need to provide at least one provider's API key:

bash scripts/dev/set-api-keys.sh <env> OPENAI_API_KEY=sk-...

This stores the key in Secrets Manager and restarts Middleman. You can set multiple keys at once:

bash scripts/dev/set-api-keys.sh <env> OPENAI_API_KEY=sk-... ANTHROPIC_API_KEY=sk-ant-...

Replace <env> with your hawk:env value (e.g., production). Supported providers: OpenAI, Anthropic, Gemini, DeepInfra, DeepSeek, Fireworks, Mistral, OpenRouter, Together, xAI.

7. Create a user (Cognito only)

If you're using the default Cognito authentication, create a user:

bash scripts/dev/create-cognito-user.sh <stack> you@example.com

The script reads the Cognito user pool from your Pulumi stack outputs, creates the user, and prints the login credentials. Skip this step if you're using your own OIDC provider.

8. Install the Hawk CLI and run your first eval

```bash uv pip install "hawk[cli] @ git+https://github.com/METR/hawk#subdirectory=hawk"

Configure the CLI to point to your deployment

uv run python scripts/dev/generate-env.py > hawk/.env

hawk login hawk eval-set hawk/examples/simple.eval-set.yaml hawk logs -f # watch it run hawk web # open results in browser ```