Contributing

Developer Setup

There are two ways to run Hawk locally:

cp hawk/.env.example hawk/.env
docker compose up --build

The defaults in .env.example are configured for fully local development (MinIO, local PostgreSQL, Minikube). For staging, update the values to point at staging infrastructure.

Then submit evals:

hawk eval-set examples/simple.eval-set.yaml

Run k9s to monitor the Inspect pod.

Full Dev Stack (API + Viewer + Live Reload)

For developing with hot reload across the full stack:

Terminal 1: Library Watch Mode

Inspect AIInspect Scout

cd ~/inspect_ai/src/inspect_ai/_view/www
pnpm install
pnpm build:lib --watch

cd ~/inspect_scout/src/inspect_scout/_view/www
pnpm install
pnpm build:lib --watch

Terminal 2: Viewer Dev Server

Update www/package.json to point to your local library, then:

cd www
pnpm install
VITE_API_BASE_URL=http://localhost:8080 pnpm dev

Terminal 3: API Server

cp hawk/.env.example hawk/.env
set -a && source hawk/.env && set +a
uv run fastapi run hawk/api/server.py --port=8080 --host=0.0.0.0 --reload

Code Quality

ruff check       # lint
ruff format      # format
basedpyright     # type check
pytest           # unit tests

All code must pass basedpyright with zero errors and zero warnings.

Testing Runner Changes

Build and push a custom runner image:

scripts/dev/build-and-push-runner-image.sh my-tag
hawk eval-set examples/simple.eval-set.yaml --image-tag my-tag

Local Minikube Setup

This runs Hawk entirely locally. It uses MinIO for S3, local PostgreSQL, a local Docker registry, and Minikube for Kubernetes. This is the same setup used by E2E tests in CI.

Prerequisites

You must be inside the devcontainer, which includes minikube, Docker-in-Docker, cilium, kubectl, helm, and gvisor.

Quick Start

These commands are run from the hawk/ directory:

cp .env.example .env
scripts/dev/start-minikube.sh

The script will:

1. Start Minikube with gvisor, containerd, and an insecure local registry
2. Create Kubernetes resources and install Cilium
3. Launch services (API server, MinIO, PostgreSQL, Docker registry)
4. Run a smoke test to verify the cluster works
5. Build and push a dummy runner image
6. Run a simple eval set to verify everything works

Running Evals Locally

HAWK_API_URL=http://localhost:8080 hawk eval-set examples/simple.eval-set.yaml --image-tag=dummy

To run real evals, build and push a real runner image:

RUNNER_IMAGE_NAME=localhost:5000/runner scripts/dev/build-and-push-runner-image.sh latest

Updating Dependencies (Inspect AI / Inspect Scout)

Use the prepare-release.py script:

# Update to a specific PyPI version
scripts/ops/prepare-release.py --inspect-ai 0.3.50

# Update to a specific git commit SHA
scripts/ops/prepare-release.py --inspect-ai abc123def456

# Update Scout
scripts/ops/prepare-release.py --inspect-scout 0.2.10

The script updates pyproject.toml files, runs uv lock, creates a release branch (for PyPI versions), and publishes any npm packages if needed.

Database Migrations

See Database for migration instructions.

Pull Requests

When creating PRs, use the template at .github/pull_request_template.md. The template includes:

• Overview and linked issue
• Approach and alternatives considered
• Testing & validation checklist
• Code quality checklist