Python client¶

Scripts can call the Hawk HTTP API without installing the CLI (no Click, no Inspect). The page below is generated from HawkClient docstrings at docs-build time.

Install¶

After a release that includes the client:

pip install "hawk[client]"
# or: uv add "hawk[client]"

From a checkout, without publishing:

pip install -e "./hawk[client]"

Task authors can read declared input identities and locations through hawk.runtime.transcript_sources(). This runner helper is separate from the HTTP client and needs no client extra.

Auth¶

HawkClient never opens a browser. Pass token=, or set HAWK_ACCESS_TOKEN, or reuse a token stored by hawk login on this machine. Set HAWK_API_URL (or pass api_url=). The client does not read or refresh a stored hawk login refresh token. Pass refresh_token= when submitting jobs that use user credentials so the runner can refresh on the cluster. With credential_mode="restricted", submission and resume requests omit the refresh token even if supplied. With user or omitted mode, they forward it; an omitted mode lets the server choose. The access token still authenticates every submission.

Dict-typed responses (get_usage, get_usage_history, and listing TypedDicts) are the server JSON. Fields may be added without a client major bump.

import asyncio
from hawk.client import HawkClient

async def main() -> None:
    async with HawkClient() as client:
        jobs = await client.get_jobs(mine=True, limit=5)
        print(jobs)

asyncio.run(main())

On a machine with no prior hawk login, pass the URL and token explicitly:

async with HawkClient(api_url="https://api.example.com", token="...") as client:
    ...

Submit an eval-set¶

Pass YAML as a dict. The server validates it. You do not need Inspect installed.

import asyncio
import pathlib

import ruamel.yaml
from hawk.client import HawkClient

async def main() -> None:
    yaml = ruamel.yaml.YAML(typ="safe")
    config = yaml.load(pathlib.Path("eval-set.yaml").read_text())
    async with HawkClient() as client:
        eval_set_id = await client.create_eval_set(config, priority="high")
        print(eval_set_id)

asyncio.run(main())

Optional local validation uses EvalSetConfig from hawk.core.types.evals (still no Inspect for typical configs). create_scan accepts a mapping the same way.

create_eval_set, create_scan, and resume_scan accept priority="low", "medium", or "high" for admission and inference. Evals default to medium; scans default to low. Each explicit resume uses its current argument or that default. Priority is a client argument, separate from the eval/scan YAML config.

Delete a job¶

await client.delete_job(job_id) waits for asynchronous restricted cleanup by retrying HTTP 503 responses. It honors Retry-After (seconds or an HTTP date), using five seconds when the header is absent or invalid. Requests and waits share the client's timeout budget, which defaults to 300 seconds. Other errors are reported immediately; only a 404 tries the scan endpoint after the eval endpoint. The hawk delete CLI command uses the same retry behavior.

API¶

HawkClient ¶

HawkClient(
    *,
    api_url: str | None = None,
    token: str | None = None,
    timeout: float = _DEFAULT_TIMEOUT_SECONDS,
)

Async HTTP client for the Hawk API.

Use as async with HawkClient(...) as client:. Requests without entering the context manager raise HawkClientError.

Pass api_url and token, or omit them to read HAWK_API_URL and reuse a token from HAWK_ACCESS_TOKEN / hawk login. Does not prompt for login or refresh a stored login token; the CLI owns that.

Submission and resume methods omit refresh_token when credential_mode="restricted". User or omitted mode forwards it for runner credential renewal; an omitted mode lets the server choose.

Open a client.

Token and URL are resolved when entering async with, not here.

Parameters:

Name	Type	Description	Default
`api_url`	`str \| None`	Hawk API base URL. Defaults to `HAWK_API_URL`.	`None`
`token`	`str \| None`	Bearer token. Defaults to `HAWK_ACCESS_TOKEN` or a `hawk login` token. Missing either raises `HawkClientError` on `async with`, not in `__init__`.	`None`
`timeout`	`float`	aiohttp total timeout in seconds for ordinary requests (default 300). Downloads and SSE streams use their own timeouts.	`_DEFAULT_TIMEOUT_SECONDS`

api_url `property` ¶

api_url: str

Resolved API base URL, without a trailing slash.

aclose `async` ¶

aclose() -> None

Close the underlying HTTP session.

create_eval_set `async` ¶

create_eval_set(
    eval_set_config: EvalSetConfig
    | Mapping[str, Any]
    | None = None,
    *,
    eval_set_id: str | None = None,
    image: str | None = None,
    image_tag: str | None = None,
    secrets: dict[str, str] | None = None,
    log_dir_allow_dirty: bool = False,
    refresh_token: str | None = None,
    allow_sensitive_cross_lab_scan: bool = False,
    force: bool = False,
    credential_mode: CredentialMode | None = None,
    priority: JobPriority = DEFAULT_EVAL_PRIORITY,
) -> EvalSetCreated

Create an eval-set, or resume one by passing eval_set_id.

Accepts an EvalSetConfig or a config mapping. Pass None with an existing id to resume the saved config. Priority defaults to medium for every submission, including resumes.

allow_sensitive_cross_lab_scan permits private transcript inputs to be sent to another provider when the deployment operator allows it. It applies only to this submission and must be re-provided on resume.

Returns:

Type	Description
`EvalSetCreated`	The eval-set id, plus any advisory warnings the server raised
`EvalSetCreated`	about the launch (empty against a server that predates them).

create_scan `async` ¶

create_scan(
    scan_config: ScanConfig | Mapping[str, Any],
    *,
    image: str | None = None,
    image_tag: str | None = None,
    secrets: dict[str, str] | None = None,
    refresh_token: str | None = None,
    allow_sensitive_cross_lab_scan: bool = False,
    force: bool = False,
    credential_mode: CredentialMode | None = None,
    priority: JobPriority = DEFAULT_SCAN_PRIORITY,
) -> str

Create a Scout scan from a ScanConfig or mapping.

Priority defaults to low.

resume_scan `async` ¶

resume_scan(
    scan_run_id: str,
    *,
    image: str | None = None,
    image_tag: str | None = None,
    secrets: dict[str, str] | None = None,
    refresh_token: str | None = None,
    allow_sensitive_cross_lab_scan: bool = False,
    force: bool = False,
    credential_mode: CredentialMode | None = None,
    priority: JobPriority = DEFAULT_SCAN_PRIORITY,
) -> str

Resume from the last checkpoint with this submission's priority.

Priority defaults to low, regardless of the original run.

plan_scan_group `async` ¶

plan_scan_group(
    scan_config: ScanConfig | Mapping[str, Any],
    *,
    shard_size: int | None = None,
) -> ScanGroupPlan

Split a scan of every eval set the caller can view into shards.

Launches nothing; launch each shard with launch_scan_group_shard. shard_size defaults to the server's (500 transcripts).

launch_scan_group_shard `async` ¶

launch_scan_group_shard(
    scan_group_id: str,
    shard: ScanGroupShard,
    scan_config: ScanConfig | Mapping[str, Any],
    *,
    shard_size: int,
    image: str | None = None,
    image_tag: str | None = None,
    secrets: dict[str, str] | None = None,
    refresh_token: str | None = None,
    allow_sensitive_cross_lab_scan: bool = False,
    force: bool = False,
    credential_mode: CredentialMode | None = None,
    priority: JobPriority = DEFAULT_SCAN_PRIORITY,
) -> ScanGroupShardLaunch

Launch one shard of a scan group plan.

Repeating a launch returns the shard's existing scan with already_launched set instead of starting another.

get_scan_group_jobs `async` ¶

get_scan_group_jobs(
    scan_group_id: str, *, limit: int = 500
) -> ScanGroupJobs

Every scan launched from a scan group, newest first, with status counts across all of them.

/meta/jobs serves at most 500 rows per page and a group can hold thousands of shards, so this walks the pages limit at a time.

get_jobs `async` ¶

get_jobs(
    *,
    mine: bool = True,
    limit: int = 10,
    status: JobStatus | None = None,
    waiting: bool = False,
) -> list[JobListItem]

List jobs (eval-sets and scans).

Parameters:

Name	Type	Description	Default
`mine`	`bool`	If true, only jobs created by the token's `sub` claim.	`True`
`limit`	`int`	Max rows.	`10`
`status`	`JobStatus \| None`	Optional status filter.	`None`
`waiting`	`bool`	If true, only jobs with pending human interactions.	`False`

get_eval_sets `async` ¶

get_eval_sets(
    *, limit: int | None = None, search: str | None = None
) -> list[EvalSetInfo]

List eval-sets.

get_evals `async` ¶

get_evals(
    eval_set_id: str, *, page: int = 1, limit: int = 100
) -> list[EvalInfo]

List evals in an eval-set.

get_samples `async` ¶

get_samples(
    eval_set_id: str,
    *,
    search: str | None = None,
    page: int = 1,
    limit: int = 50,
) -> list[SampleListItem]

List one page of samples in an eval-set.

get_all_samples_for_eval_set `async` ¶

get_all_samples_for_eval_set(
    eval_set_id: str, *, limit: int | None = None
) -> list[SampleListItem]

Walk sample pages until exhausted, or until limit rows.

search_transcripts `async` ¶

search_transcripts(
    query: str,
    *,
    eval_set_id: str | None = None,
    sort: Literal["rank", "recency"] = "rank",
    limit: int = 50,
    cursor: str | None = None,
    diversify: bool = True,
) -> SearchResultPage

One page of full-text transcript search (GET /meta/search).

Pass the returned next_cursor back as cursor, with the same query, scope, sort and diversify, until it is None. Cursors expire after one hour. diversify=False lifts the viewer's per-transcript hit cap.

stream_search_csv `async` ¶

stream_search_csv(
    query: str, *, eval_set_id: str | None = None
) -> AsyncIterator[bytes]

Stream every full-text match as CSV (GET /meta/search.csv).

One request; the server walks the pages and flushes each as rows. If it stops early the last line is #resume-cursor=<cursor>; resume with search_transcripts(..., sort="recency", diversify=False, cursor=...).

get_me `async` ¶

get_me() -> dict[str, Any]

The caller's identity as the server resolves it.

Returns the GET /auth/me JSON: sub, email (the claim named by the deployment's email field, or None), exp, is_admin, model_groups (each with read, write and execute flags) and code_groups, as this deployment enforces them.

get_usage `async` ¶

get_usage(group_by: str = 'user') -> list[dict[str, Any]]

Current token usage, grouped by provider and model.

group_by selects the split: "user" (the default, filling each model's users), "team" (teams), "job" (jobs) or "task" (tasks). The parameter is only sent when it differs from the default, so servers predating it see the same request as before.

Returns the server JSON list. Fields may be added without a client major bump.

get_usage_history `async` ¶

get_usage_history(
    start: int,
    end: int,
    bin_seconds: int,
    group_by: str = "user",
) -> list[dict[str, Any]]

Binned token usage between Unix timestamps start and end.

group_by selects the split: "user" (the default, filling each bin's users), "team" (teams), "job" (jobs) or "task" (tasks). The parameter is only sent when it differs from the default, so servers predating it see the same request as before.

Returns the server JSON list. Fields may be added without a client major bump.

get_log_files `async` ¶

get_log_files(eval_set_id: str) -> list[LogFileInfo]

List .eval log files for an eval-set.

get_log_headers `async` ¶

get_log_headers(file_names: list[str]) -> list[EvalHeader]

Fetch partial eval-log headers for the given file names.

get_download_url `async` ¶

get_download_url(log_path: str) -> tuple[str, str]

Presign one log file.

Returns:

Type	Description
`tuple[str, str]`	`(url, filename)`. Fetch the URL yourself; this is not an S3 client.

get_download_urls `async` ¶

get_download_urls(
    log_paths: list[str],
) -> AsyncIterator[tuple[str, str]]

Yield presigned S3 URLs for log files as each batch completes.

Batches of BATCH_DOWNLOAD_URLS_LIMIT paths are posted concurrently.

download_to_file `async` ¶

download_to_file(path: str, destination: Path) -> None

GET an API path and write the body to destination.

download_scan_export `async` ¶

download_scan_export(
    scanner_result_uuid: str, destination: Path
) -> str

Download a scan-export CSV.

Returns:

Type	Description
`str`	Server filename from `Content-Disposition`.

get_sample_metadata `async` ¶

get_sample_metadata(sample_uuid: str) -> SampleMetadata

Look up where a sample's eval log lives.

list_sample_artifacts `async` ¶

list_sample_artifacts(
    eval_set_id: str, sample_uuid: str
) -> BrowseResponse

List files in a sample's artifact tree.

get_sample_artifact_file_url `async` ¶

get_sample_artifact_file_url(
    eval_set_id: str, sample_uuid: str, artifact_path: str
) -> PresignedUrlResponse

Presign one sample artifact file.

fetch_logs `async` ¶

fetch_logs(
    job_id: str,
    *,
    since: datetime | None = None,
    limit: int | None = 100,
    sort: SortOrder = DESC,
    from_start: bool = False,
) -> list[LogEntry]

Fetch runner logs for a job.

get_job_monitoring_data `async` ¶

get_job_monitoring_data(
    job_id: str, *, since: datetime | None = None
) -> JobMonitoringData

Pod/status snapshot for a job.

get_eval_set_status `async` ¶

get_eval_set_status(job_id: str) -> EvalSetStatus

One-shot eval-set progress snapshot.

stream_eval_set_status `async` ¶

stream_eval_set_status(
    job_id: str,
) -> AsyncIterator[EvalSetStatus]

SSE stream of eval-set status until the connection ends.

stop_eval_set `async` ¶

stop_eval_set(
    eval_set_id: str,
    *,
    sample_uuid: str | None = None,
    error: bool = False,
) -> None

Stop an eval-set, or one sample when sample_uuid is set.

delete_job `async` ¶

delete_job(job_id: str) -> Literal['eval set', 'scan']

Delete an eval set or scan, retrying 503s within the client timeout.

Honor Retry-After, defaulting to five seconds. Only 404 falls back to scan.

get_job_status `async` ¶

get_job_status(job_id: str) -> JobStatusResponse | None

Job status, or None if the job is not found.

HawkClientError ¶

Bases: Exception

Base error for :class:~hawk.client.HawkClient.

Raised for missing config or token, and for connection failures.

HawkAPIError ¶

HawkAPIError(status: int, message: str)

Bases: HawkClientError

The Hawk API returned a non-success HTTP status.

Attributes:

Name	Type	Description
`status`	`int`	HTTP status code.
`message`	`str`	Error text from the API body or reason phrase.

Python client¶

Install¶

Auth¶

Submit an eval-set¶

Delete a job¶

API¶

HawkClient ¶

api_url property ¶

aclose async ¶

create_eval_set async ¶

create_scan async ¶

resume_scan async ¶

plan_scan_group async ¶

launch_scan_group_shard async ¶

get_scan_group_jobs async ¶

get_jobs async ¶

get_eval_sets async ¶

get_evals async ¶

get_samples async ¶

get_all_samples_for_eval_set async ¶

search_transcripts async ¶

stream_search_csv async ¶

get_me async ¶

get_usage async ¶

get_usage_history async ¶

get_log_files async ¶

get_log_headers async ¶

get_download_url async ¶

get_download_urls async ¶

download_to_file async ¶

download_scan_export async ¶

get_sample_metadata async ¶

list_sample_artifacts async ¶

get_sample_artifact_file_url async ¶

fetch_logs async ¶

get_job_monitoring_data async ¶

get_eval_set_status async ¶

stream_eval_set_status async ¶

stop_eval_set async ¶

delete_job async ¶

get_job_status async ¶

HawkClientError ¶

HawkAPIError ¶

api_url `property` ¶

aclose `async` ¶

create_eval_set `async` ¶

create_scan `async` ¶

resume_scan `async` ¶

plan_scan_group `async` ¶

launch_scan_group_shard `async` ¶

get_scan_group_jobs `async` ¶

get_jobs `async` ¶

get_eval_sets `async` ¶

get_evals `async` ¶

get_samples `async` ¶

get_all_samples_for_eval_set `async` ¶

search_transcripts `async` ¶

stream_search_csv `async` ¶

get_me `async` ¶

get_usage `async` ¶

get_usage_history `async` ¶

get_log_files `async` ¶

get_log_headers `async` ¶

get_download_url `async` ¶

get_download_urls `async` ¶

download_to_file `async` ¶

download_scan_export `async` ¶

get_sample_metadata `async` ¶

list_sample_artifacts `async` ¶

get_sample_artifact_file_url `async` ¶

fetch_logs `async` ¶

get_job_monitoring_data `async` ¶

get_eval_set_status `async` ¶

stream_eval_set_status `async` ¶

stop_eval_set `async` ¶

delete_job `async` ¶

get_job_status `async` ¶