Contributing

Developer Setup

There are two ways to run Hawk locally:

bash cp hawk/.env.example hawk/.env docker compose up --build

The defaults in .env.example are configured for fully local development (MinIO, local PostgreSQL, Minikube). For staging, update the values to point at staging infrastructure.

Then submit evals:

bash hawk eval-set examples/simple.eval-set.yaml

Run k9s to monitor the Inspect pod.

Full Dev Stack (API + Viewer + Live Reload)

For developing with hot reload across the full stack:

Terminal 1: Library Watch Mode

Inspect AIInspect Scout

bash cd ~/inspect_ai/src/inspect_ai/_view/www yarn install yarn build:lib --watch

bash cd ~/inspect_scout/src/inspect_scout/_view/www pnpm install pnpm build:lib --watch

Terminal 2: Viewer Dev Server

Update www/package.json to point to your local library, then:

bash cd www yarn install VITE_API_BASE_URL=http://localhost:8080 yarn dev

Terminal 3: API Server

bash cp hawk/.env.example hawk/.env set -a && source hawk/.env && set +a uv run fastapi run hawk/api/server.py --port=8080 --host=0.0.0.0 --reload

Code Quality

bash ruff check # lint ruff format # format basedpyright # type check pytest # unit tests

All code must pass basedpyright with zero errors and zero warnings.

Testing Runner Changes

Build and push a custom runner image:

bash scripts/dev/build-and-push-runner-image.sh my-tag hawk eval-set examples/simple.eval-set.yaml --image-tag my-tag

Local Minikube Setup

This runs Hawk entirely locally. It uses MinIO for S3, local PostgreSQL, a local Docker registry, and Minikube for Kubernetes. This is the same setup used by E2E tests in CI.

Prerequisites

You must be inside the devcontainer, which includes minikube, Docker-in-Docker, cilium, kubectl, helm, and gvisor.

Quick Start

These commands are run from the hawk/ directory:

bash cp .env.example .env scripts/dev/start-minikube.sh

The script will:

1. Start Minikube with gvisor, containerd, and an insecure local registry
2. Create Kubernetes resources and install Cilium
3. Launch services (API server, MinIO, PostgreSQL, Docker registry)
4. Run a smoke test to verify the cluster works
5. Build and push a dummy runner image
6. Run a simple eval set to verify everything works

Running Evals Locally

bash HAWK_API_URL=http://localhost:8080 hawk eval-set examples/simple.eval-set.yaml --image-tag=dummy

To run real evals, build and push a real runner image:

bash RUNNER_IMAGE_NAME=localhost:5000/runner scripts/dev/build-and-push-runner-image.sh latest

Updating Dependencies (Inspect AI / Inspect Scout)

Use the prepare-release.py script:

```bash

Update to a specific PyPI version

scripts/ops/prepare-release.py --inspect-ai 0.3.50

Update to a specific git commit SHA

scripts/ops/prepare-release.py --inspect-ai abc123def456

Update Scout

scripts/ops/prepare-release.py --inspect-scout 0.2.10 ```

The script updates pyproject.toml files, runs uv lock, creates a release branch (for PyPI versions), and publishes any npm packages if needed.

Database Migrations

See Database for migration instructions.

Pull Requests

When creating PRs, use the template at .github/pull_request_template.md. The template includes:

• Overview and linked issue
• Approach and alternatives considered
• Testing & validation checklist
• Code quality checklist