Python client¶

Scripts can call the Hawk HTTP API without installing the CLI (no Click, no Inspect). The page below is generated from HawkClient docstrings at docs-build time.

Install¶

After a release that includes the client:

pip install "hawk[client]"
# or: uv add "hawk[client]"

From a checkout, without publishing:

pip install -e "./hawk[client]"

Auth¶

HawkClient never opens a browser. Pass token=, or set HAWK_ACCESS_TOKEN, or reuse a token stored by hawk login on this machine. Set HAWK_API_URL (or pass api_url=). The client does not read or refresh a stored hawk login refresh token. Long-running evals can still pass refresh_token= into create_eval_set / create_scan so the runner can refresh on the cluster.

Dict-typed responses (get_usage, get_usage_history, and listing TypedDicts) are the server JSON. Fields may be added without a client major bump.

import asyncio
from hawk.client import HawkClient

async def main() -> None:
    async with HawkClient() as client:
        jobs = await client.get_jobs(mine=True, limit=5)
        print(jobs)

asyncio.run(main())

On a machine with no prior hawk login, pass the URL and token explicitly:

async with HawkClient(api_url="https://api.example.com", token="...") as client:
    ...

Submit an eval-set¶

Pass YAML as a dict. The server validates it. You do not need Inspect installed.

import asyncio
import pathlib

import ruamel.yaml
from hawk.client import HawkClient

async def main() -> None:
    yaml = ruamel.yaml.YAML(typ="safe")
    config = yaml.load(pathlib.Path("eval-set.yaml").read_text())
    async with HawkClient() as client:
        eval_set_id = await client.create_eval_set(config)
        print(eval_set_id)

asyncio.run(main())

Optional local validation uses EvalSetConfig from hawk.core.types.evals (still no Inspect for typical configs). create_scan accepts a mapping the same way.

API¶

HawkClient ¶

HawkClient(
    *,
    api_url: str | None = None,
    token: str | None = None,
    timeout: float = _DEFAULT_TIMEOUT_SECONDS,
)

Async HTTP client for the Hawk API.

Use as async with HawkClient(...) as client:. Requests without entering the context manager raise HawkClientError.

Pass api_url and token, or omit them to read HAWK_API_URL and reuse a token from HAWK_ACCESS_TOKEN / hawk login. Does not prompt for login or refresh a stored login token; the CLI owns that.

Open a client.

Token and URL are resolved when entering async with, not here.

Parameters:

Name	Type	Description	Default
`api_url`	`str \| None`	Hawk API base URL. Defaults to `HAWK_API_URL`.	`None`
`token`	`str \| None`	Bearer token. Defaults to `HAWK_ACCESS_TOKEN` or a `hawk login` token. Missing either raises `HawkClientError` on `async with`, not in `__init__`.	`None`
`timeout`	`float`	aiohttp total timeout in seconds for ordinary requests (default 300). Downloads and SSE streams use their own timeouts.	`_DEFAULT_TIMEOUT_SECONDS`

api_url `property` ¶

api_url: str

Resolved API base URL, without a trailing slash.

aclose `async` ¶

aclose() -> None

Close the underlying HTTP session.

create_eval_set `async` ¶

create_eval_set(
    eval_set_config: EvalSetConfig
    | Mapping[str, Any]
    | None = None,
    *,
    eval_set_id: str | None = None,
    image: str | None = None,
    image_tag: str | None = None,
    secrets: dict[str, str] | None = None,
    log_dir_allow_dirty: bool = False,
    refresh_token: str | None = None,
) -> str

Create an eval-set, or resume one by passing eval_set_id.

Parameters:

Name	Type	Description	Default
`eval_set_config`	`EvalSetConfig \| Mapping[str, Any] \| None`	Eval-set YAML as a mapping, or an `EvalSetConfig`. Omit when resuming. The server validates the document; this client does not import Inspect.	`None`
`eval_set_id`	`str \| None`	Existing id to resume. Server restores config from S3.	`None`
`image`	`str \| None`	Optional runner image override.	`None`
`image_tag`	`str \| None`	Optional runner image tag override.	`None`
`secrets`	`dict[str, str] \| None`	Runner secrets to inject.	`None`
`log_dir_allow_dirty`	`bool`	Allow a non-empty log dir.	`False`
`refresh_token`	`str \| None`	OIDC refresh token for the runner, when needed. The client does not refresh a `hawk login` token itself.	`None`

Returns:

Type	Description
`str`	The eval-set id.

create_scan `async` ¶

create_scan(
    scan_config: ScanConfig | Mapping[str, Any],
    *,
    image: str | None = None,
    image_tag: str | None = None,
    secrets: dict[str, str] | None = None,
    refresh_token: str | None = None,
    allow_sensitive_cross_lab_scan: bool = False,
) -> str

Create a Scout scan.

Parameters:

Name	Type	Description	Default
`scan_config`	`ScanConfig \| Mapping[str, Any]`	Scan YAML as a mapping, or a `ScanConfig`.	required
`image`	`str \| None`	Optional runner image override.	`None`
`image_tag`	`str \| None`	Optional runner image tag override.	`None`
`secrets`	`dict[str, str] \| None`	Runner secrets to inject.	`None`
`refresh_token`	`str \| None`	OIDC refresh token for the runner, when needed.	`None`
`allow_sensitive_cross_lab_scan`	`bool`	Skip the cross-lab scan check.	`False`

Returns:

Type	Description
`str`	The scan run id.

resume_scan `async` ¶

resume_scan(
    scan_run_id: str,
    *,
    image: str | None = None,
    image_tag: str | None = None,
    secrets: dict[str, str] | None = None,
    refresh_token: str | None = None,
    allow_sensitive_cross_lab_scan: bool = False,
) -> str

Resume a scan from its last checkpoint.

Returns:

Type	Description
`str`	The scan run id.

get_jobs `async` ¶

get_jobs(
    *,
    mine: bool = True,
    limit: int = 10,
    status: JobStatus | None = None,
    waiting: bool = False,
) -> list[JobListItem]

List jobs (eval-sets and scans).

Parameters:

Name	Type	Description	Default
`mine`	`bool`	If true, only jobs created by the token's `sub` claim.	`True`
`limit`	`int`	Max rows.	`10`
`status`	`JobStatus \| None`	Optional status filter.	`None`
`waiting`	`bool`	If true, only jobs with pending human interactions.	`False`

get_eval_sets `async` ¶

get_eval_sets(
    *, limit: int | None = None, search: str | None = None
) -> list[EvalSetInfo]

List eval-sets.

get_evals `async` ¶

get_evals(
    eval_set_id: str, *, page: int = 1, limit: int = 100
) -> list[EvalInfo]

List evals in an eval-set.

get_samples `async` ¶

get_samples(
    eval_set_id: str,
    *,
    search: str | None = None,
    page: int = 1,
    limit: int = 50,
) -> list[SampleListItem]

List one page of samples in an eval-set.

get_all_samples_for_eval_set `async` ¶

get_all_samples_for_eval_set(
    eval_set_id: str, *, limit: int | None = None
) -> list[SampleListItem]

Walk sample pages until exhausted, or until limit rows.

get_usage `async` ¶

get_usage() -> list[dict[str, Any]]

Current token usage, grouped by provider and model.

Returns the server JSON list. Fields may be added without a client major bump.

get_usage_history `async` ¶

get_usage_history(
    start: int, end: int, bin_seconds: int
) -> list[dict[str, Any]]

Binned token usage between Unix timestamps start and end.

Returns the server JSON list. Fields may be added without a client major bump.

get_log_files `async` ¶

get_log_files(eval_set_id: str) -> list[LogFileInfo]

List .eval log files for an eval-set.

get_log_headers `async` ¶

get_log_headers(file_names: list[str]) -> list[EvalHeader]

Fetch partial eval-log headers for the given file names.

get_download_url `async` ¶

get_download_url(log_path: str) -> tuple[str, str]

Presign one log file.

Returns:

Type	Description
`tuple[str, str]`	`(url, filename)`. Fetch the URL yourself; this is not an S3 client.

get_download_urls `async` ¶

get_download_urls(
    log_paths: list[str],
) -> AsyncIterator[tuple[str, str]]

Yield presigned S3 URLs for log files as each batch completes.

Batches of BATCH_DOWNLOAD_URLS_LIMIT paths are posted concurrently.

download_to_file `async` ¶

download_to_file(path: str, destination: Path) -> None

GET an API path and write the body to destination.

download_scan_export `async` ¶

download_scan_export(
    scanner_result_uuid: str, destination: Path
) -> str

Download a scan-export CSV.

Returns:

Type	Description
`str`	Server filename from `Content-Disposition`.

get_sample_metadata `async` ¶

get_sample_metadata(sample_uuid: str) -> SampleMetadata

Look up where a sample's eval log lives.

list_sample_artifacts `async` ¶

list_sample_artifacts(
    eval_set_id: str, sample_uuid: str
) -> BrowseResponse

List files in a sample's artifact tree.

get_sample_artifact_file_url `async` ¶

get_sample_artifact_file_url(
    eval_set_id: str, sample_uuid: str, artifact_path: str
) -> PresignedUrlResponse

Presign one sample artifact file.

fetch_logs `async` ¶

fetch_logs(
    job_id: str,
    *,
    since: datetime | None = None,
    limit: int | None = 100,
    sort: SortOrder = DESC,
    from_start: bool = False,
) -> list[LogEntry]

Fetch runner logs for a job.

get_job_monitoring_data `async` ¶

get_job_monitoring_data(
    job_id: str, *, since: datetime | None = None
) -> JobMonitoringData

Pod/status snapshot for a job.

get_eval_set_status `async` ¶

get_eval_set_status(job_id: str) -> EvalSetStatus

One-shot eval-set progress snapshot.

stream_eval_set_status `async` ¶

stream_eval_set_status(
    job_id: str,
) -> AsyncIterator[EvalSetStatus]

SSE stream of eval-set status until the connection ends.

stop_eval_set `async` ¶

stop_eval_set(
    eval_set_id: str,
    *,
    sample_uuid: str | None = None,
    error: bool = False,
) -> None

Stop an eval-set, or one sample when sample_uuid is set.

delete_job `async` ¶

delete_job(job_id: str) -> Literal['eval set', 'scan']

Delete an eval set or scan run. Tries eval-set first, then scan.

get_job_status `async` ¶

get_job_status(job_id: str) -> JobStatusResponse | None

Job status, or None if the job is not found.

HawkClientError ¶

Bases: Exception

Base error for :class:~hawk.client.HawkClient.

Raised for missing config or token, and for connection failures.

HawkAPIError ¶

HawkAPIError(status: int, message: str)

Bases: HawkClientError

The Hawk API returned a non-success HTTP status.

Attributes:

Name	Type	Description
`status`	`int`	HTTP status code.
`message`	`str`	Error text from the API body or reason phrase.

Python client¶

Install¶

Auth¶

Submit an eval-set¶

API¶

HawkClient ¶

api_url property ¶

aclose async ¶

create_eval_set async ¶

create_scan async ¶

resume_scan async ¶

get_jobs async ¶

get_eval_sets async ¶

get_evals async ¶

get_samples async ¶

get_all_samples_for_eval_set async ¶

get_usage async ¶

get_usage_history async ¶

get_log_files async ¶

get_log_headers async ¶

get_download_url async ¶

get_download_urls async ¶

download_to_file async ¶

download_scan_export async ¶

get_sample_metadata async ¶

list_sample_artifacts async ¶

get_sample_artifact_file_url async ¶

fetch_logs async ¶

get_job_monitoring_data async ¶

get_eval_set_status async ¶

stream_eval_set_status async ¶

stop_eval_set async ¶

delete_job async ¶

get_job_status async ¶

HawkClientError ¶

HawkAPIError ¶

api_url `property` ¶

aclose `async` ¶

create_eval_set `async` ¶

create_scan `async` ¶

resume_scan `async` ¶

get_jobs `async` ¶

get_eval_sets `async` ¶

get_evals `async` ¶

get_samples `async` ¶

get_all_samples_for_eval_set `async` ¶

get_usage `async` ¶

get_usage_history `async` ¶

get_log_files `async` ¶

get_log_headers `async` ¶

get_download_url `async` ¶

get_download_urls `async` ¶

download_to_file `async` ¶

download_scan_export `async` ¶

get_sample_metadata `async` ¶

list_sample_artifacts `async` ¶

get_sample_artifact_file_url `async` ¶

fetch_logs `async` ¶

get_job_monitoring_data `async` ¶

get_eval_set_status `async` ¶

stream_eval_set_status `async` ¶

stop_eval_set `async` ¶

delete_job `async` ¶

get_job_status `async` ¶