Managing Your Deployment¶
Day-to-day Commands¶
bash
pulumi up # deploy changes
pulumi preview # preview without deploying
pulumi stack output --json # view outputs (API URL, DB endpoint, etc.)
pulumi refresh # sync Pulumi state with actual AWS resources
Updating Hawk¶
Pull the latest code and redeploy:
bash
git pull
cd infra
pulumi up
Database migrations run automatically during deployment.
Updating Inspect AI or Scout¶
Use the release preparation script:
bash
scripts/ops/prepare-release.py --inspect-ai 0.3.50 # specific PyPI version
scripts/ops/prepare-release.py --inspect-ai abc123 # specific git commit
scripts/ops/prepare-release.py --inspect-scout 0.2.10 # update Scout
Then deploy and run smoke tests to validate.
Smoke Tests¶
Validate that a deployed environment is working end-to-end:
bash
hawk login
scripts/dev/smoke # test current stack
scripts/dev/smoke --stack my-org # test a specific stack
scripts/dev/smoke --warehouse # include database checks
scripts/dev/smoke -k test_real_llm # run a specific test
Smoke tests submit real evals against real models and verify results end up in the warehouse and viewer.
After updating Inspect AI or Scout dependencies:
bash
uv run pytest hawk/tests/smoke -m smoke --smoke -n 10 -vv
Tearing Down¶
bash
pulumi destroy # delete all AWS resources
pulumi stack rm # remove the stack from Pulumi state
Warning
Always wait for pulumi destroy to finish before running stack rm. Running stack rm first will orphan AWS resources in your account.
Troubleshooting¶
Pulumi State Issues¶
```bash
See what Pulumi thinks exists vs what's actually in AWS¶
pulumi refresh
If a resource is stuck, remove it from state (doesn't delete from AWS)¶
pulumi state delete '
Import an existing AWS resource into Pulumi state¶
pulumi import aws:ec2/securityGroup:SecurityGroup my-sg sg-0123456789 ```
Redeploying a Single Resource¶
```bash
pulumi up --target 'urn:pulumi:dev-
Tip: run pulumi stack export to find resource URNs¶
```