Configuration Reference¶

All infrastructure configuration lives in Pulumi.<stack-name>.yaml. See Pulumi.example.yaml for a fully documented reference with all available options.

Required Settings¶

Config Key	Description	Example
`hawk:domain`	Root domain — Hawk prepends `hawk.` to form service URLs (e.g. `api.hawk.example.com`)	`example.com`
`hawk:publicDomain`	Public domain for DNS zones	`example.com`
`hawk:primarySubnetCidr`	VPC CIDR block	`10.0.0.0/16`

Domain & DNS¶

Hawk's services live under hawk.<hawk:domain> — for example, with hawk:domain: example.com the API is at api.hawk.example.com (Pulumi adds the hawk. segment automatically; do not include it in the config value). DNS delegation must work before ACM TLS certificates can validate. Four paths — pick one before deploying:

How hawk:domain and hawk:publicDomain must relate

Service hostnames are built as *.hawk.<hawk:domain> and the load balancer's certificates cover platform.<hawk:domain> and *.hawk.<hawk:domain> — and both certificates validate via DNS records in the hawk:publicDomain Route 53 zone. So hawk:domain must equal or be a subdomain of hawk:publicDomain; anything else puts the validation records in a zone that can't serve them, and cert validation hangs (preflight checks this).

When you delegate only a single subdomain (e.g. a Cloudflare parent with just hawk.example.com delegated to Route 53), set both values to that subdomain — hawk:domain: hawk.example.com and hawk:publicDomain: hawk.example.com. Service URLs then look like api.hawk.hawk.example.com (the doubled hawk is cosmetic and expected).

Option	When	What to set
A. Route 53 Domains	New setup, simplest path. AWS handles registration, hosted zone, and delegation in a single step.	`hawk:createPublicZone: "false"` (the default) — Pulumi looks up the existing zone created by registration.
B. Existing registrar + manual delegation	You already own the domain at Namecheap/GoDaddy/etc. and want to keep it there.	Create/import the Route 53 public hosted zone first, update your registrar to its four NS records, and leave `hawk:createPublicZone: "false"`.
C. Cloudflare delegation	Parent domain is in Cloudflare and you want a subdomain delegated to AWS.	Create/import the Route 53 public hosted zone first, then create NS records in Cloudflare before deploy.
D. HTTP-only (testing only)	Smoke-testing without a real domain.	`hawk:skipTlsCerts: "true"`. Services reachable only via the raw ALB DNS name. Not for real use.

Cert-validation hang gotcha

If DNS isn't working when pulumi up runs, the wildcard ACM certificate validation will hang for ~75 minutes (default timeout) before failing. Get delegation in place first and run scripts/dev/preflight.sh before deploy.

Leave createPublicZone disabled for strict preflight

scripts/dev/preflight.sh validates that hawk:publicDomain is already delegated to the expected Route 53 hosted zone before pulumi up. If hawk:createPublicZone: "true", Pulumi would create a new zone whose NS records cannot be known or delegated before the run, so strict preflight fails. Create/import the public hosted zone first, delegate DNS, and leave createPublicZone: "false".

Authentication¶

When hawk:oidcClientId is not set, Hawk provisions a Cognito user pool during pulumi up and uses it as the auth provider. Create your first user with scripts/dev/create-cognito-user.sh <stack> <email> after the deploy finishes.

To use your own OIDC provider (Okta, Auth0, etc.), set all three of these — Hawk will skip the Cognito setup:

Config Key	Description	Example
`hawk:oidcClientId`	OIDC client ID	`your-client-id`
`hawk:oidcAudience`	OIDC audience for access tokens	`https://api.example.com`
`hawk:oidcIssuer`	OIDC issuer URL	`https://login.example.com/oauth2/default`

Quota increases (sizing Path A)¶

The Quick Start's sizing section offers two paths for fresh AWS accounts; this is the detail for Path A (raising the quotas to fit Hawk's default footprint). Request all three on day one — approval can take days:

# Fargate On-Demand vCPUs (Hawk needs ≥ ~9 for a comfortable default deploy)
aws service-quotas request-service-quota-increase \
  --service-code fargate --quota-code L-3032A538 --desired-value 32 --region <region>
# EC2 Standard On-Demand vCPUs (runner nodes)
aws service-quotas request-service-quota-increase \
  --service-code ec2 --quota-code L-1216C47A --desired-value 32 --region <region>
# Fargate Spot vCPUs (the eval-log importer Batch job needs 8)
aws service-quotas request-service-quota-increase \
  --service-code fargate --quota-code L-36FBB829 --desired-value 32 --region <region>

Brand-new accounts are often denied ("use 90% of your current capacity, then re-request after the next billing cycle"). If that happens, escalate through your AWS account team or AWS Sales contact — they can approve increases that automated support won't — or start with Path B from the Quick Start sizing section in the meantime. The commands may also fail immediately with QuotaExceededException: maximum number of open service quota increase (SQI) requests — brand-new accounts can already be at the open-request cap from AWS's own auto-filed requests; that's fine, proceed on Path B and re-request once the pre-existing requests resolve.

Sizing beyond the defaults¶

The --desired-value 32 above is a try-out number. For real workloads, the quota that scales with usage is EC2 Standard On-Demand — each concurrent eval consumes a runner node plus its sandbox pods (typically 4-8 vCPUs together), on top of a ~6-vCPU cluster baseline. Fargate does not scale with eval count (it runs only the fixed services and importer jobs), so ~32 Fargate vCPUs is enough for any deployment size; a high Fargate quota simply stops it from ever being the constraint.

Rough tiers to anchor a request:

Tier	Fargate OD / Spot	EC2 Standard On-Demand	G/VT On-Demand (GPU)
Try-out (Path B)	4 (default)	8 (default)	0 — set `hawk:enableGpuOperator: "false"`
Small team (~5 concurrent evals)	32	64	64 if you run GPU tasks
Production-scale eval org	1,000+	1,000-6,000	256-768

AWS approves increases step-wise: request 2-4× your current need, generate usage, and re-request as you grow — jumping straight from 8 to thousands will be denied. The production-scale column is what a mature account with billing history can reach, usually via the account-team route rather than the self-service console.

GPU quotas¶

If your evals request GPUs, you also need the G and VT instance quota — its fresh-account default is 0, and Hawk's default GPU NodePool uses on-demand capacity:

# G/VT On-Demand vCPUs (g4dn GPU nodes; a g4dn.xlarge is 4 vCPUs)
aws service-quotas request-service-quota-increase \
  --service-code ec2 --quota-code L-DB2E81BA --desired-value 64 --region <region>

GPU quota increases are the slowest and most-denied category — request them well before you need them, and expect anything beyond ~100 vCPUs to require your AWS account team. Note that with the quota at 0, a task requesting a GPU doesn't fail — Karpenter retries provisioning silently forever and the pod stays Pending; if you don't run GPU tasks, set hawk:enableGpuOperator: "false" and skip this entirely.

Infrastructure Options¶

Config Key	Default	Description
`hawk:eksK8sVersion`	`1.35`	Kubernetes version for EKS
`hawk:eksPublicEndpoint`	`true`	Set to `false` to make the EKS API endpoint private (requires Tailscale or VPN to reach `kubectl`)
`hawk:albIdleTimeout`	`3600`	ALB idle timeout in seconds
`hawk:albInternal`	`false`	Set to `true` to make the ALB internal (requires VPN)
`hawk:cloudwatchLogsRetentionDays`	`14`	CloudWatch log retention
`hawk:excludeZoneIds`	`[]`	AZ zone IDs (e.g. `use1-az3`) to exclude from VPC/EKS subnets
`hawk:dbMinCapacity`	`0`	Aurora Serverless v2 minimum ACUs (0 lets the cluster auto-pause)
`hawk:dbMaxCapacity`	`8`	Aurora Serverless v2 maximum ACUs
`hawk:protectResources`	`true` (non-dev), `false` (dev envs)	When `true`, S3 buckets and secrets are protected from accidental deletion (Pulumi protect + `force_destroy=false`). Unset defaults to protected except on dev envs; set explicitly to override.
`hawk:runnerMemory`	`16Gi` (app default)	Per-runner-pod memory limit. Raise for large eval sets.
`hawk:scanImporterMemorySize`	`3008`	Scan importer Lambda memory in MB
`hawk:scanImporterEphemeralStorageSize`	`3008`	Scan importer Lambda ephemeral storage in MB
`hawk:enableS3ObjectLambda`	`false`	Create S3 Object Lambda Access Points for permission-filtered eval log reads
`hawk:publicModelsOnly`	`false`	Lock the deployment to public-group model data only — token broker forces `model-access-public`, OLAP restricts to public groups, RLS bypass is removed from non-admin DB users
`hawk:highAvailability`	`false`	Run 3 replicas of stateless services (API, Middleman) across AZs instead of 1. Middleman HA requires Valkey (`valkeyEnabled` or `valkeyUrl`).

EKS rejects control-plane subnets in unsupported AZs

By default, all AZs reported as available for the region are used for the VPC and EKS control-plane subnets. Some AZs don't support every service — EKS in particular fails with UnsupportedAvailabilityZoneException: Cannot create cluster ... in us-east-1e. Retry cluster creation using control plane subnets that span at least two of these availability zones: us-east-1a, .... Add the rejected zone's zone ID (e.g. use1-az3 for us-east-1e — zone IDs, not zone names, are stable across accounts) to hawk:excludeZoneIds:

config:
  hawk:excludeZoneIds:
    - use1-az3

Verify against the live AWS error message — supported AZs change over time.

Authentication (advanced)¶

The three keys in the Authentication section above are the minimum for an external OIDC provider. These additional keys override defaults that are otherwise discovered from the issuer:

Config Key	Default	Description
`hawk:oidcScopes`	`openid profile email offline_access`	Scopes requested at login. Space-separated.
`hawk:oidcEmailField`	`email`	JWT claim used to extract the user's email.
`hawk:oidcJwksUri`	discovered	Override the JWKS URI (rare).
`hawk:oidcTokenEndpoint`	discovered	Override the token endpoint.
`hawk:oidcAuthorizationEndpoint`	discovered	Override the authorization endpoint.
`hawk:oidcDeviceAuthorizationEndpoint`	discovered	Override for the device code flow.
`hawk:oidcRevocationEndpoint`	discovered	Override for token revocation.
`hawk:defaultPermissions`	`model-access-public`	Space-separated permissions granted to authenticated users whose tokens carry no `permissions` / `scp` claim (Cognito users by default).

Run the OIDC autodiscovery script

python scripts/dev/discover-oidc.py <issuer-url> <client-id> <audience> prints the full set of hawk:oidc* config values to paste into your stack file.

Observability¶

Config Key	Default	Description
`hawk:enableDatadog`	`false`	Enable Datadog metrics, logs, and APM. See Datadog below.
`hawk:datadogSite`	`datadoghq.com`	Datadog site (e.g. `datadoghq.eu`, `us3.datadoghq.com`).
`hawk:createDatadogAwsIntegration`	`false`	When `true`, Pulumi provisions the Datadog ↔ AWS integration role.
`hawk:hawkSentryDsn`	unset	Sentry DSN for the Hawk API + Lambdas.
`hawk:middlemanSentryDsn`	unset	Sentry DSN for Middleman.
`hawk:otelTracingEnabled`	`false`	Send OTLP traces from API/Middleman/Lambdas to AWS X-Ray.
`hawk:otelSpanSamplingPercentage`	`100`	Sampling rate (0–100) when OTel tracing is enabled.
`hawk:stuckEvalWebhookUrl`	unset	HTTPS webhook subscribed to the stuck-eval-set SNS alert topic (prd-only monitor for eval-sets whose samples stop progressing).

Optional Integrations¶

These are all disabled by default. Enable them in your stack config when needed.

Datadog¶

Monitoring, APM, and log forwarding:

hawk:enableDatadog: "true"
hawk:datadogSite: datadoghq.com

Requires a <env>/platform/datadog-api-key secret in AWS Secrets Manager.

Cloudflare¶

Option C from the Domain & DNS table above, for when your parent domain is in Cloudflare and you want a subdomain delegated to AWS. In both cases hawk:publicDomain must be a subdomain of the Cloudflare parent domain.

There are two mutually exclusive ways to do this — pick one:

Manual delegation (strict-preflight compatible, recommended)¶

Leave hawk:createPublicZone: "false" and do not set the hawk:cloudflare* keys. Create or import the Route 53 public hosted zone first, then create NS records in your Cloudflare parent zone (pointing the subdomain at that zone's four nameservers) before deploy. If the subdomain was delegated before (e.g. an earlier deployment attempt), replace the old NS records — adding new ones alongside stale ones yields a mixed nameserver set and intermittent SERVFAIL. No Cloudflare API token is needed, and scripts/dev/preflight.sh can verify the delegation before pulumi up.

When only this single subdomain is delegated to Route 53, set both hawk:publicDomain and hawk:domain to it (e.g. both hawk.example.com) — service certs validate in the publicDomain zone, so hawk:domain must live inside it; the resulting api.hawk.hawk.example.com-style URLs are cosmetic. See the domain-relationship warning in Domain & DNS above.

Pulumi-managed delegation (incompatible with strict preflight)¶

Let Pulumi create the Route 53 public zone and manage the Cloudflare NS records for you. This requires hawk:createPublicZone: "true", so the zone's nameservers don't exist until the run — strict preflight will fail the DNS check (see the createPublicZone warning above).

hawk:publicDomain must be a subdomain of hawk:cloudflareParentDomain (Pulumi raises a ValueError otherwise).

hawk:createPublicZone: "true"
hawk:cloudflareZoneId: "your-zone-id"
hawk:cloudflareParentDomain: "example.com"
hawk:publicDomain: "hawk.example.com"

Before pulumi up, create a Cloudflare API token (Zone:DNS:Edit on the parent zone) and store it in AWS Secrets Manager:

aws secretsmanager create-secret \
  --name "<env>/platform/cloudflare-api-token" \
  --secret-string "<token>"

<env> defaults to your Pulumi stack name. The deploy will fail with a "secret not found" error if this isn't set up first.

Tailscale¶

VPN overlay for private service access:

Set hawk:albInternal: "true" and store a Tailscale auth key in AWS Secrets Manager. This makes all services accessible only through your Tailscale network.

CrowdStrike Falcon¶

Endpoint protection for EKS nodes and the Tailscale subnet router:

hawk:enableCrowdstrike: "true"

Requires a <env>/platform/crowdstrike secret in AWS Secrets Manager with:

{
  "cid": "YOUR-CUSTOMER-ID",
  "client_id": "YOUR-API-CLIENT-ID",
  "client_secret": "YOUR-API-CLIENT-SECRET",
  "base_url": "https://api.us-2.crowdstrike.com"
}

Setup:

In the CrowdStrike Falcon console, go to Support and resources > API clients and keys and create a client with Sensor Download: Read scope.
Copy your CID from Host setup and management > Deploy > Sensor downloads.
Your base_url matches your Falcon console URL (e.g. falcon.us-2.crowdstrike.com → https://api.us-2.crowdstrike.com).

When enabled, this installs the Falcon sensor on:

GPU nodes (AL2023) — via the Karpenter EC2NodeClass userData at instance boot
Tailscale subnet router (AL2023 ARM64) — via cloud-init at instance boot

The falcon-sensor DaemonSet for all EKS nodes (including Bottlerocket) requires the Falcon Images Download API scope, which is part of the Falcon Cloud Security with Containers add-on.

Budget Alerts¶

hawk:budgetLimit: "10000"
hawk:budgetNotificationEmails:
  - "team@example.com"

When integrations are disabled, services fall back to simpler alternatives (CloudWatch instead of Datadog, no DNS delegation, etc.).