Inspect AI and Scout Forks¶

Hawk installs inspect-ai and inspect-scout from PyPI when an upstream release has everything it needs. When it needs a fix that upstream hasn't released, it pins a commit on METR's fork instead (METR/inspect_ai, METR/inspect_scout, and METR/ts-mono for the viewers both of them embed). This page covers how to change what those forks carry. For the release script itself, see Updating Dependencies.

Rules¶

Drift is temporary. Every patch a fork carries is meant to land upstream, and Hawk goes back to a plain upstream release once a release contains them all.
Every carried patch is tracked by a PR. Open it upstream (UKGovernmentBEIS/inspect_ai, meridianlabs-ai/inspect_scout, meridianlabs-ai/ts-mono), at least as a draft. If you can't open PRs there, open it against the METR fork instead. Open the PR before the patch lands on a fork branch Hawk pins.
The pin lists what it carries. The comment above each fork's entry in [tool.uv.sources] in hawk/pyproject.toml names the upstream release the pin is based on and links every carried PR, with ts-mono's under a ts-mono (viewer): heading.
Fork-only housekeeping needs no PR: ts-mono submodule bumps, the Scout LFS strip, and inspect_ai's sandbox-tools binaries.

Branches¶

Branch	Purpose
`hawk-pin/<date>-<description>`	One per pinned commit. It keeps the commit reachable; don't delete one that a released Hawk version pins.
`hotfix`	The newest fork commit. Force-push it whenever you publish a new one. Nothing installs from it.
`release/<timestamp>`	Pushed by `prepare-release.py` on each bump.
`hawk-wheels`	An orphan branch with one empty commit. Every wheel release's tag points at it (see below).

Keep pinned commits reachable with branches, not tags, on inspect_scout: it builds with hatch-vcs and no git describe filter, so the nearest tag becomes the version, and a hawk-pin/... tag fails the build. inspect_ai matches only N.N.N tags, so its tags are harmless, but use branches there too.

Upgrading to a new upstream release¶

The same steps add a patch to the current pin, except that you branch from the current pin instead of an upstream tag.

Check what the current pin carries. In a checkout of the fork, list the commits between the upstream tag the pin is based on and the pin (git log --oneline <tag>..<pin>), and the same range in the ts-mono submodule. For each carried PR, check whether the target release contains it. Anything it doesn't contain must be carried forward or deliberately dropped.
Build the branch. Fetch upstream's tags (git fetch --tags --force upstream; they have no v prefix), check out the target tag and cherry-pick the patches still needed with git cherry-pick -x, squashing a multi-commit PR into one commit so the branch shows one commit per PR. Patches the release already contains come up empty: drop them. Redo the housekeeping commits rather than cherry-picking them.
inspect_scout: strip Git LFS. Upstream stores the built viewer in LFS, and a fork doesn't share upstream's LFS storage, so cloning the fork fails when Git LFS tries to download those files. On every new branch:
```
git lfs untrack 'src/inspect_scout/_view/dist/**'
git rm .gitattributes
find src/inspect_scout/_view/dist -type f -exec touch {} \;
git add src/inspect_scout/_view/dist/ .gitattributes
git commit -m "chore: remove LFS tracking, store dist files as regular git objects"
```
git lfs ls-files must then print nothing. 4. ts-mono, if a viewer patch is carried. Base a METR/ts-mono branch on the newer of the two ts-mono commits the upstream releases pin, cherry-pick the viewer PRs onto it, push it as a hawk-pin/... branch, and point each fork's src/<package>/_view/ts-mono submodule at it. 5. inspect_ai: rebuild the sandbox tools if the sandbox-tools source or its build inputs changed (see below). 6. Push. Push upstream's release tags to the fork too (git push origin <tag>): setuptools-scm derives the version from them. Then push the branch as hawk-pin/<date>-<description>. 7. Test Hawk against it. Point the entry in [tool.uv.sources] at your checkout ({ path = "/absolute/path/to/inspect_ai" }), run uv lock and uv sync --all-extras in hawk/, then run the tests. Hawk's API returns Inspect's event models, so a changed model also shows up in the API compatibility check. 8. Publish. Run scripts/ops/prepare-release.py with the commit SHA, then scripts/dev/uv-lock-all.sh, and update the comment above the pin. The script builds the fork's wheel, attaches it to a GitHub release on the fork and pins its URL, so it needs gh logged in with push access to the fork; it also publishes the viewer to npm, which needs npm login. 9. Move hotfix on each fork you changed to the new commit (git push origin +<sha>:refs/heads/hotfix).

Wheel releases¶

Hawk pins each fork as a wheel attached to a GitHub release on the fork, so installs download one wheel instead of cloning the fork and building it. prepare-release.py creates the release. Its tag is hawk-wheels/<timestamp>-<commit>, with the commit's first 12 hex digits, and points at the hawk-wheels orphan branch rather than the pinned commit, so no build ever sees it. Both forks have immutable releases turned on: a published wheel can't be replaced or deleted.

To move a commit that is already pinned as a git source onto its wheel, pass --wheel-only: it publishes and pins the wheel without rebuilding the viewer or touching hawk/www/package.json.

Wheels build with the backend versions pinned in scripts/ops/fork-wheel-build-constraints.txt. The fork-wheels job in Hawk CI rebuilds every pinned wheel from the commit its tag names and fails if any file differs. Changing a pinned backend version changes the files it generates, so republish the pinned wheels along with it.

Sandbox-tools binaries¶

inspect_ai's fork changes the sandbox-tools source, so it can't use upstream's published binaries. It commits its own to src/inspect_ai/binaries/ and numbers its builds in the 9000s, so they are never mistaken for, or fetched as, upstream's. tests/tools/sandbox_tools_utils/test_fork_bundled_binaries.py checks the binaries against SHA256SUMS, checks that each contains the current package source, and checks SOURCE_SHA256, a digest of the source and build inputs, so it fails as soon as either changes without a rebuild.

To rebuild, in a checkout of the fork:

Set src/inspect_ai/tool/_sandbox_tools_utils/sandbox_tools_version.txt to the next unused build number (one more than the last) and commit that on its own. Never reuse a number.

Build all four binaries with Docker. Each architecture builds fastest on a host of that architecture:

uv run python src/inspect_ai/tool/_sandbox_tools_utils/build_within_container.py --arch amd64 --dev=false
uv run python src/inspect_ai/tool/_sandbox_tools_utils/build_within_container.py --arch amd64 --musl --dev=false
uv run python src/inspect_ai/tool/_sandbox_tools_utils/build_within_container.py --arch arm64 --dev=false
uv run python src/inspect_ai/tool/_sandbox_tools_utils/build_within_container.py --arch arm64 --musl --dev=false

Record the digests and the source they were built from:

(cd src/inspect_ai/binaries && sha256sum inspect-sandbox-tools-*-v<N>) \
  > src/inspect_ai/tool/_sandbox_tools_utils/SHA256SUMS
uv run python tests/tools/sandbox_tools_utils/test_fork_bundled_binaries.py
uv run pytest tests/tools/sandbox_tools_utils/test_fork_bundled_binaries.py

Remove the previous build's binaries and commit the new ones with the two sums files. src/inspect_ai/binaries/ is gitignored and the binaries exceed the large-file hook's limit, so add them with git add -f and commit with --no-verify.