Skip to content

New Environment Setup

Prerequisites

  • AWS account with appropriate permissions
  • Pulumi CLI installed
  • AWS CLI configured
  • A domain name for your services

Do not deploy to us-east-1

Two us-east-1-specific failure modes (the us-east-1e AZ being EKS-unsupported, and the legacy ec2.internal DNS suffix tripping Bottlerocket's pluto) cause EKS nodes to never join the cluster — no evals can run. Use us-west-2 (project default) or an EU region like eu-west-1 / eu-central-1. This warning should be removed once both issues are fixed upstream.

Quick Start

1. Install prerequisites

brew install pulumi awscli uv python@3.13 jq

Install Pulumi, uv, the AWS CLI, Python 3.13+, and jq.

2. Authenticate

aws sso login --profile your-profile
# or: aws configure

3. Set up Pulumi state backend

Create an S3 bucket and KMS key:

aws s3 mb s3://my-org-hawk-pulumi-state  # must be globally unique
aws kms create-alias --alias-name alias/pulumi-secrets \
  --target-key-id $(aws kms create-key --query KeyMetadata.KeyId --output text)
pulumi login s3://my-org-hawk-pulumi-state

Credential troubleshooting

If pulumi login fails with NoCredentialProviders, your AWS credentials aren't visible to Pulumi. Make sure you ran aws configure (not just aws login). If using SSO profiles, ensure AWS_PROFILE is set, or export credentials explicitly:

eval "$(aws configure export-credentials --format env)"

4. Choose a domain and DNS strategy

Hawk's services run on subdomains of hawk:domain (e.g. api.hawk.example.com), each protected by an ACM TLS certificate. Pulumi needs a Route 53 public hosted zone for hawk:publicDomain and DNS delegation working before certs can validate. Pick one path before configuring the stack:

  • (a) Route 53 Domains — AWS handles registration, zone creation, and delegation. Leave hawk:createPublicZone: "false" (the default).
  • (b) Existing registrar + manual delegation — Set hawk:createPublicZone: "true", deploy, then paste the four NS records from the new Route 53 zone into your registrar.
  • (c) Cloudflare automatic delegation — Parent zone in Cloudflare; Hawk creates NS records there for you. See the Configuration Reference for the full setup (including the required Secrets Manager entry).
  • (d) Skip TLS — testing only — Set hawk:skipTlsCerts: "true" for an HTTP-only deploy.

See the Configuration Reference: Domain & DNS for full details. Note: with options (b) or (c), if DNS isn't working when pulumi up runs, the wildcard ACM cert validation will hang for ~75 min before failing — get delegation in place first.

5. Create and deploy

cd infra
pulumi stack init my-org --secrets-provider="awskms://alias/pulumi-secrets"
cp ../Pulumi.example.yaml ../Pulumi.my-org.yaml
# Edit Pulumi.my-org.yaml with your values (including the createPublicZone setting from step 4)
docker login          # required — Docker Hub (https://hub.docker.com/) rate-limits anonymous pulls
docker login dhi.io   # required — Hawk's Python base images live on dhi.io (free Community tier; same Docker Hub credentials)
pulumi up

First deploy creates ~200+ AWS resources and takes about 15-20 minutes.

6. Secrets

For production stacks, set up API keys:

scripts/dev/set-api-keys.sh <env> OPENAI_API_KEY=sk-...

For dev environments, secrets are automatically shared from staging — no manual seeding needed.

To enable runners to clone private GitHub repos, configure the git credentials secret:

scripts/dev/set-git-config.sh <env> <github-pat>

The token is stored in Secrets Manager and readable by anyone with access. Use a fine-grained PAT with minimal scope.

7. Your services

After deployment, your services are available at:

  • API: https://api.hawk.<domain>
  • Viewer: https://viewer.hawk.<domain>
  • Middleman: middleman-ecs.<domain>

8. View API logs

aws logs tail "$(pulumi stack output api_log_group_name)" \
  --region us-west-2 --since 30m --format short | grep -v /health

Dev Environments

For development, create lightweight environments that share staging infrastructure:

./scripts/dev/new-dev-env.sh <your-name>    # e.g., ./scripts/dev/new-dev-env.sh alice

Requires PULUMI_BACKEND_URL to be exported and a deployed stg stack in that backend (the script clones its config). The script configures the stack and prompts to deploy. Your dev environment shares staging's VPC, ALB, and EKS cluster while getting its own database and services.

See Deployment for more details on managing dev environments.

Tailscale VPN Setup

If using Tailscale for private service access:

  1. Pick an IPv4 CIDR that doesn't conflict with existing ranges (e.g., 10.13.0.0/16)
  2. Add the CIDR to your Tailscale ACL rules
  3. Add a custom nameserver for <environment>.<your-domain> pointing to the CIDR's DNS resolver (e.g., 10.13.0.2)
  4. Set hawk:albInternal: "true" in your stack config
  5. Store a Tailscale auth key in AWS Secrets Manager
  6. Deploy with pulumi up