Python client¶

Scripts can call the Hawk HTTP API without installing the CLI (no Click, no Inspect). The page below is generated from HawkClient docstrings at docs-build time.

Install¶

After a release that includes the client:

pip install "hawk[client]"
# or: uv add "hawk[client]"

From a checkout, without publishing:

pip install -e "./hawk[client]"

Auth¶

HawkClient never opens a browser. Pass token=, or set HAWK_ACCESS_TOKEN, or reuse a token stored by hawk login on this machine. Set HAWK_API_URL (or pass api_url=). The client does not read or refresh a stored hawk login refresh token. Long-running evals can still pass refresh_token= into create_eval_set / create_scan so the runner can refresh on the cluster.

Dict-typed responses (get_usage, get_usage_history, and listing TypedDicts) are the server JSON. Fields may be added without a client major bump.

import asyncio
from hawk.client import HawkClient

async def main() -> None:
    async with HawkClient() as client:
        jobs = await client.get_jobs(mine=True, limit=5)
        print(jobs)

asyncio.run(main())

On a machine with no prior hawk login, pass the URL and token explicitly:

async with HawkClient(api_url="https://api.example.com", token="...") as client:
    ...

Submit an eval-set¶

Pass YAML as a dict. The server validates it. You do not need Inspect installed.

import asyncio
import pathlib

import ruamel.yaml
from hawk.client import HawkClient

async def main() -> None:
    yaml = ruamel.yaml.YAML(typ="safe")
    config = yaml.load(pathlib.Path("eval-set.yaml").read_text())
    async with HawkClient() as client:
        eval_set_id = await client.create_eval_set(config, priority="high")
        print(eval_set_id)

asyncio.run(main())

Optional local validation uses EvalSetConfig from hawk.core.types.evals (still no Inspect for typical configs). create_scan accepts a mapping the same way.

create_eval_set, create_scan, and resume_scan accept priority="low", "medium", or "high" for admission and inference. Evals default to medium; scans default to low. Each explicit resume uses its current argument or that default. Priority is a client argument, separate from the eval/scan YAML config.

API¶

HawkClient ¶

HawkClient(
    *,
    api_url: str | None = None,
    token: str | None = None,
    timeout: float = _DEFAULT_TIMEOUT_SECONDS,
)

Async HTTP client for the Hawk API.

Use as async with HawkClient(...) as client:. Requests without entering the context manager raise HawkClientError.

Pass api_url and token, or omit them to read HAWK_API_URL and reuse a token from HAWK_ACCESS_TOKEN / hawk login. Does not prompt for login or refresh a stored login token; the CLI owns that.

Open a client.

Token and URL are resolved when entering async with, not here.

Parameters:

Name	Type	Description	Default
`api_url`	`str \| None`	Hawk API base URL. Defaults to `HAWK_API_URL`.	`None`
`token`	`str \| None`	Bearer token. Defaults to `HAWK_ACCESS_TOKEN` or a `hawk login` token. Missing either raises `HawkClientError` on `async with`, not in `__init__`.	`None`
`timeout`	`float`	aiohttp total timeout in seconds for ordinary requests (default 300). Downloads and SSE streams use their own timeouts.	`_DEFAULT_TIMEOUT_SECONDS`

api_url `property` ¶

api_url: str

Resolved API base URL, without a trailing slash.

aclose `async` ¶

aclose() -> None

Close the underlying HTTP session.

create_eval_set `async` ¶

create_eval_set(
    eval_set_config: EvalSetConfig
    | Mapping[str, Any]
    | None = None,
    *,
    eval_set_id: str | None = None,
    image: str | None = None,
    image_tag: str | None = None,
    secrets: dict[str, str] | None = None,
    log_dir_allow_dirty: bool = False,
    refresh_token: str | None = None,
    force: bool = False,
    priority: JobPriority = DEFAULT_EVAL_PRIORITY,
) -> EvalSetCreated

Create an eval-set, or resume one by passing eval_set_id.

Accepts an EvalSetConfig or a config mapping. Pass None with an existing id to resume the saved config. Priority defaults to medium for every submission, including resumes.

refresh_token lets the runner refresh its credentials on the cluster; this client does not refresh a stored hawk login token.

Returns:

Type	Description
`EvalSetCreated`	The eval-set id, plus any advisory warnings the server raised
`EvalSetCreated`	about the launch (empty against a server that predates them).

create_scan `async` ¶

create_scan(
    scan_config: ScanConfig | Mapping[str, Any],
    *,
    image: str | None = None,
    image_tag: str | None = None,
    secrets: dict[str, str] | None = None,
    refresh_token: str | None = None,
    allow_sensitive_cross_lab_scan: bool = False,
    force: bool = False,
    priority: JobPriority = DEFAULT_SCAN_PRIORITY,
) -> str

Create a Scout scan from a ScanConfig or mapping.

Priority defaults to low. refresh_token lets the cluster runner refresh its credentials; this client does not refresh stored tokens.

resume_scan `async` ¶

resume_scan(
    scan_run_id: str,
    *,
    image: str | None = None,
    image_tag: str | None = None,
    secrets: dict[str, str] | None = None,
    refresh_token: str | None = None,
    allow_sensitive_cross_lab_scan: bool = False,
    force: bool = False,
    priority: JobPriority = DEFAULT_SCAN_PRIORITY,
) -> str

Resume from the last checkpoint with this submission's priority.

Priority defaults to low, regardless of the original run.

get_jobs `async` ¶

get_jobs(
    *,
    mine: bool = True,
    limit: int = 10,
    status: JobStatus | None = None,
    waiting: bool = False,
) -> list[JobListItem]

List jobs (eval-sets and scans).

Parameters:

Name	Type	Description	Default
`mine`	`bool`	If true, only jobs created by the token's `sub` claim.	`True`
`limit`	`int`	Max rows.	`10`
`status`	`JobStatus \| None`	Optional status filter.	`None`
`waiting`	`bool`	If true, only jobs with pending human interactions.	`False`

get_eval_sets `async` ¶

get_eval_sets(
    *, limit: int | None = None, search: str | None = None
) -> list[EvalSetInfo]

List eval-sets.

get_evals `async` ¶

get_evals(
    eval_set_id: str, *, page: int = 1, limit: int = 100
) -> list[EvalInfo]

List evals in an eval-set.

get_samples `async` ¶

get_samples(
    eval_set_id: str,
    *,
    search: str | None = None,
    page: int = 1,
    limit: int = 50,
) -> list[SampleListItem]

List one page of samples in an eval-set.

get_all_samples_for_eval_set `async` ¶

get_all_samples_for_eval_set(
    eval_set_id: str, *, limit: int | None = None
) -> list[SampleListItem]

Walk sample pages until exhausted, or until limit rows.

get_me `async` ¶

get_me() -> dict[str, Any]

The caller's identity as the server resolves it.

Returns the GET /auth/me JSON: sub, email (the claim named by the deployment's email field, or None) and exp.

get_usage `async` ¶

get_usage(group_by: str = 'user') -> list[dict[str, Any]]

Current token usage, grouped by provider and model.

group_by selects the split: "user" (the default, filling each model's users), "team" (teams), "job" (jobs) or "task" (tasks). The parameter is only sent when it differs from the default, so servers predating it see the same request as before.

Returns the server JSON list. Fields may be added without a client major bump.

get_usage_history `async` ¶

get_usage_history(
    start: int,
    end: int,
    bin_seconds: int,
    group_by: str = "user",
) -> list[dict[str, Any]]

Binned token usage between Unix timestamps start and end.

group_by selects the split: "user" (the default, filling each bin's users), "team" (teams), "job" (jobs) or "task" (tasks). The parameter is only sent when it differs from the default, so servers predating it see the same request as before.

Returns the server JSON list. Fields may be added without a client major bump.

get_log_files `async` ¶

get_log_files(eval_set_id: str) -> list[LogFileInfo]

List .eval log files for an eval-set.

get_log_headers `async` ¶

get_log_headers(file_names: list[str]) -> list[EvalHeader]

Fetch partial eval-log headers for the given file names.

get_download_url `async` ¶

get_download_url(log_path: str) -> tuple[str, str]

Presign one log file.

Returns:

Type	Description
`tuple[str, str]`	`(url, filename)`. Fetch the URL yourself; this is not an S3 client.

get_download_urls `async` ¶

get_download_urls(
    log_paths: list[str],
) -> AsyncIterator[tuple[str, str]]

Yield presigned S3 URLs for log files as each batch completes.

Batches of BATCH_DOWNLOAD_URLS_LIMIT paths are posted concurrently.

download_to_file `async` ¶

download_to_file(path: str, destination: Path) -> None

GET an API path and write the body to destination.

download_scan_export `async` ¶

download_scan_export(
    scanner_result_uuid: str, destination: Path
) -> str

Download a scan-export CSV.

Returns:

Type	Description
`str`	Server filename from `Content-Disposition`.

get_sample_metadata `async` ¶

get_sample_metadata(sample_uuid: str) -> SampleMetadata

Look up where a sample's eval log lives.

list_sample_artifacts `async` ¶

list_sample_artifacts(
    eval_set_id: str, sample_uuid: str
) -> BrowseResponse

List files in a sample's artifact tree.

get_sample_artifact_file_url `async` ¶

get_sample_artifact_file_url(
    eval_set_id: str, sample_uuid: str, artifact_path: str
) -> PresignedUrlResponse

Presign one sample artifact file.

fetch_logs `async` ¶

fetch_logs(
    job_id: str,
    *,
    since: datetime | None = None,
    limit: int | None = 100,
    sort: SortOrder = DESC,
    from_start: bool = False,
) -> list[LogEntry]

Fetch runner logs for a job.

get_job_monitoring_data `async` ¶

get_job_monitoring_data(
    job_id: str, *, since: datetime | None = None
) -> JobMonitoringData

Pod/status snapshot for a job.

get_eval_set_status `async` ¶

get_eval_set_status(job_id: str) -> EvalSetStatus

One-shot eval-set progress snapshot.

stream_eval_set_status `async` ¶

stream_eval_set_status(
    job_id: str,
) -> AsyncIterator[EvalSetStatus]

SSE stream of eval-set status until the connection ends.

stop_eval_set `async` ¶

stop_eval_set(
    eval_set_id: str,
    *,
    sample_uuid: str | None = None,
    error: bool = False,
) -> None

Stop an eval-set, or one sample when sample_uuid is set.

delete_job `async` ¶

delete_job(job_id: str) -> Literal['eval set', 'scan']

Delete an eval set or scan run. Tries eval-set first, then scan.

get_job_status `async` ¶

get_job_status(job_id: str) -> JobStatusResponse | None

Job status, or None if the job is not found.

HawkClientError ¶

Bases: Exception

Base error for :class:~hawk.client.HawkClient.

Raised for missing config or token, and for connection failures.

HawkAPIError ¶

HawkAPIError(status: int, message: str)

Bases: HawkClientError

The Hawk API returned a non-success HTTP status.

Attributes:

Name	Type	Description
`status`	`int`	HTTP status code.
`message`	`str`	Error text from the API body or reason phrase.

Python client¶

Install¶

Auth¶

Submit an eval-set¶

API¶

HawkClient ¶

api_url property ¶

aclose async ¶

create_eval_set async ¶

create_scan async ¶

resume_scan async ¶

get_jobs async ¶

get_eval_sets async ¶

get_evals async ¶

get_samples async ¶

get_all_samples_for_eval_set async ¶

get_me async ¶

get_usage async ¶

get_usage_history async ¶

get_log_files async ¶

get_log_headers async ¶

get_download_url async ¶

get_download_urls async ¶

download_to_file async ¶

download_scan_export async ¶

get_sample_metadata async ¶

list_sample_artifacts async ¶

get_sample_artifact_file_url async ¶

fetch_logs async ¶

get_job_monitoring_data async ¶

get_eval_set_status async ¶

stream_eval_set_status async ¶

stop_eval_set async ¶

delete_job async ¶

get_job_status async ¶

HawkClientError ¶

HawkAPIError ¶

api_url `property` ¶

aclose `async` ¶

create_eval_set `async` ¶

create_scan `async` ¶

resume_scan `async` ¶

get_jobs `async` ¶

get_eval_sets `async` ¶

get_evals `async` ¶

get_samples `async` ¶

get_all_samples_for_eval_set `async` ¶

get_me `async` ¶

get_usage `async` ¶

get_usage_history `async` ¶

get_log_files `async` ¶

get_log_headers `async` ¶

get_download_url `async` ¶

get_download_urls `async` ¶

download_to_file `async` ¶

download_scan_export `async` ¶

get_sample_metadata `async` ¶

list_sample_artifacts `async` ¶

get_sample_artifact_file_url `async` ¶

fetch_logs `async` ¶

get_job_monitoring_data `async` ¶

get_eval_set_status `async` ¶

stream_eval_set_status `async` ¶

stop_eval_set `async` ¶

delete_job `async` ¶

get_job_status `async` ¶