Skip to content

Web Viewer

Hawk includes a web UI for browsing and analyzing evaluation results, served at viewer.hawk.<domain>.

Opening the Viewer

hawk web                  # open current eval set in browser
hawk web EVAL_SET_ID      # open a specific eval set
hawk view-sample UUID     # open a specific sample

Features

The web viewer supports:

  • Browsing eval sets and scan runs — see all submitted evaluations
  • Filtering samples — by score range, completion-time range, and full-text search
  • Comparing results — across eval sets and models
  • Exporting to CSV — download filtered results
  • Shareable URLs — filter state is encoded in query params for easy sharing
  • Jobs — your launched jobs (or everyone's) with live status, status and approval filters, and an owner-only stop control that escalates to force stop
  • Rate limits & token usage — /usage opens in Live for current rate limits. Historic defaults to the past week, with a combined chart, model details, and rankings by user, team, run or task. Filter individual users and models, select or deselect all, and bookmark the URL to save the view. See Token Usage for controls, endpoints and label semantics.
  • Execution timelines — per-sample timeline pages with tool-call deep links and duration histograms (requires tracing to be enabled)
  • Access and admin — what you can view, edit and call, and, for Hawk admins, code-access rules with a preview of each change (see Access and admin)

Open a job or eval-set name to read its transcripts. Eval Sets also supports checkbox selection for comparing samples across several sets. Jobs filters and Eval Sets/Scans search and pagination are stored in the URL; the return link in Inspect and Scout brings you back to that list. Eval Sets also keeps checkbox selection within the current browser tab.

Switching between Hawk pages keeps the navigation in place. Jobs, Eval Sets, Scans and live Usage reuse recently loaded results when you return; older results remain visible while they refresh. Slower updates show a progress indicator, and a failed background refresh is reported alongside the last loaded results.

Run pages link to Transcripts, Status, Timeline, Samples and Search for the same eval set. New eval set opens the launch form from any Hawk page; Clone prefills it from a run's configuration with a new ID. If the source configuration cannot be loaded, retry or explicitly start from the template before launching.

Access and admin

Open Settings from the account menu. My access shows your model groups, with whether you can view results, edit results and call each group's models, and your code-access groups. It reports what this deployment enforces, so a public-models-only deployment shows only the public group. Open a model group to see its models. Longer lists are counted per company; open a company, search, or use Show all to list them all. Open a code group to see the source prefixes of the rules that require it.

Hawk admins also see an Admin group with Code access:

  • Rules lists code-access rules with how many recorded sources need each rule's group and how many runs use them. Open a rule to list its sources (up to 50 in full; longer lists open by kind, git or package, or search, or use Show all), and a source to see its tasks, solvers, scorers and other code items. Default (the * rule) comes first, even before it exists: fill in a group and Set… it to require that group of every source no other rule matches; unset, those sources are unrestricted. Can’t classify holds older spellings no rule can match, whose runs stay hidden from non-admins.
  • Creating, changing or deleting a rule opens a preview first: which recorded sources would change group (counted per group when there are more than 50), how many runs use them, and a warning when the group is new to Hawk: only people your identity provider grants it can use those sources, admins included. Nothing is saved until you confirm, and a delete asks once more in a browser dialog.

The API checks admin status on every request; the page only hides what it would refuse.

Opening logs in other viewers

Deployments can offer other viewers besides the built-in Inspect viewer: web apps served by Hawk, other web hosts, or desktop apps. When any are configured, an Open in menu appears next to eval sets, logs and samples, listing each viewer that can open that item.

Choose Set default on a viewer to open items with it when you click their names. The preference is stored per browser, and items the default viewer can't open use the built-in one. Launches and sample permalinks never open a desktop app automatically. Admins configure viewers as described in Viewer Plug-ins.

Branding

The header shows Hawk's wordmark. The viewer bundles marks for known organizations, keyed by the stack's hawk:org (passed to the image build as VITE_ORG), and shows the matching one in front of the wordmark; any other value leaves the header organization-neutral. Adding one means adding an SVG under hawk/www/src/assets/ and an entry in the ORG_MARKS table in Layout.tsx.

Before the first log file

While an eval set starts, its viewer shows the latest 500 available runner log entries, including retained entries older than 24 hours: errors in red, warnings in amber, and structured exception tracebacks. The latest error also appears above the logs while it remains in that window.

Job status determines the heading and stops the spinner when the run ends. Reported failures get a persistent banner; status polling continues through retry gaps so a recovered job can update the page. Startup trouble is also surfaced while the job is active, with distinct headings for stuck jobs and capacity waits. A log error alone does not mark the job failed. Use View job status for pod health and startup details.

If admission status is unknown while the job is pending, the viewer shows Status unavailable and keeps polling for its current status.

Loaded logs remain visible through unavailable or empty responses after runner cleanup. This copy lives only in the open page; refreshing or reopening requires the backend to still have the logs. Missing-log messages explain runner removal and retention expiry. Server retention and saved evaluation files are unchanged.

Live eval-set status

The Status view gives an at-a-glance, drill-down picture of an entire eval set while it is still running — useful for spotting stuck pods, missing images (ImagePullBackOff), rate-limit storms, and slow sandbox calls.

Open it by clicking an eval set's status in the jobs list, choosing Status on a run page, or navigating to /eval-set/<EVAL_SET_ID>/status. It updates live (polling every few seconds) and shows:

  • Overall status, runner pod health (CPU/memory, sandbox pod/GPU counts), and a segmented progress bar (completed / running / errored / limit / waiting).
  • Per-task progress bars with trouble badges (e.g. image pull failing).
  • Per-sample phase (waiting / init / running / scoring / completed / errored / limit), retries, limit hit, latest score, token usage, and the last event with both relative and UTC time plus how long it took.
  • A "Waiting for Kubernetes" list of samples whose sandbox pod has not been scheduled yet, with the reason (e.g. Unschedulable) and the scheduler message.

If you created the eval set, the view (and its row in the jobs list) offers Stop, which scores partial work and lets the run exit; the same dialog escalates to Force stop, which tears down the run's Kubernetes resources immediately. Both are owner-only.

Access is scoped: you only see eval sets whose model groups you have access to (same as the rest of the viewer), and only the creator can stop or delete a run.

Live data is read directly from Kubernetes (pod status/metrics) and from inspect_ai's shared sample buffer in S3, so it is available before the eval set has been imported into the warehouse. It is served by GET /monitoring/jobs/{eval_set_id}/eval-set-status.

Review tool approvals

When a run is waiting for human approval, open its Approvals tab. The Status page also links there and shows the number waiting. You can share /eval-set/<run-id>/approvals directly. This works in a mobile browser: sign in and choose View waiting samples (or Review approvals if no pending count is reported). A single waiting tool approval opens directly; otherwise use the searchable sample table, with the same columns as the ACP terminal picker. It shows 50 samples per page; search covers all samples. Back to samples returns to that table without answering the current request. The panel shows complete, syntax-highlighted tool arguments, the monitor that escalated the action and its explanation when available, and the agent's preceding message. Expand Conversation context for earlier events; it opens at the latest messages so you can scroll upward. Inspect's original approval prompt is also preserved there. If structured tool arguments are unavailable, that prompt appears directly in the review instead. If several tools need approval, Next request moves to the next one after the current request resolves. Open full transcript lets you inspect earlier context beyond ACP's recent-event replay. If several evaluations match the sample, Open run logs appears instead to avoid choosing the wrong transcript. The panel keeps the latest 100 context entries and up to 16,000 characters per entry, and shows a notice when context is omitted or shortened. Tool arguments are kept complete; if the runner does not provide them, the panel says they are unavailable. Questions and forms still require hawk acp in a terminal.

Approve runs the command. Reject blocks it so the agent can continue. Terminate sample asks for confirmation, then blocks it and ends only the current sample. Reconnect reloads the current request after a connection problem; it never answers it.

Anyone with content access and write access to every model group on the run can answer approvals in the web viewer, matching hawk acp's model-group write requirement. Ownership or admin status is not required. Read-only credentials cannot approve. Decision sent means the response was transmitted; Approved · confirmed by the runner (or Rejected) confirms the matching decision and remains visible when the run finishes. Closing the panel leaves an unanswered request pending until someone answers or the run's approval timeout expires. Reopening retrieves a fresh request.

Hawk audits decisions under the signed-in account. This does not verify that a human, rather than automation using that account, answered.

The run must have acp_server enabled and a human approval policy; see Babysitting Evals. Inspect sends each request to one connected client at a time, so a request already open in a terminal does not appear in the viewer; disconnect that client and reconnect the viewer. Questions, forms, modifying tool arguments, and custom human escalation choices still use the terminal.

Searching across transcripts

The Transcript Search page searches across the eval sets and scans you have permission to read. An optional eval-set filter narrows the results and, in literal mode, the records searched in each batch.

  • Full text finds words and quoted phrases and can order results by relevance or recency. Words match across inflections (verify finds verified and verifying). Common function words such as the or it are ignored, so a sentence finds its content words and a quoted phrase still requires the remaining words at the same distance. A query that contains a negation word (no, not, nor, never, without, against, cannot or a contraction such as don't) counts every word, so "did not comply" does not match did comply. Model names, file paths and version strings match whole; a URL matches by its host or path. While transcripts are being re-indexed the page says so ("The search index may be incomplete") and the affected records match on exact words only. A word or phrase that matches too much of the corpus to rank within the search budget shows its newest matches instead, under a notice with the approximate match count and how to narrow the search (the API returns the same as too_broad).
  • Literal text finds the text you enter, including punctuation and whitespace. Capitalization is ignored by default; enable Match case to require the same capitalization. Quotes are ordinary characters; leave them out unless they are part of the text you want to find. Results arrive newest-first. Both modes read the same document: for structured event payloads and message content parts that is their text values, not the JSON keys or syntax around them.

Literal search works through history newest-first, about four seconds of searching per request. The page header shows how far back the search has reached ("searched back to ..."), counted in records you can read. Use Continue searching to search older records, including when a request finds no matches. The search is finished only when there is no continuation. Results are ordered by when Hawk imported the record, which may differ from when the evaluation ran. A full search of a large corpus can still take many requests, and a request can time out on unusually large records or a busy database.

This mode searches the existing searchable warehouse fields, including message text, event content and evaluation metadata. It does not resolve separately stored attachments, and some large structured fields are truncated to the warehouse's existing search limits. A completed search means those fields have been checked, not that every byte of every original transcript has been read. Use in-transcript grep below for attachment-aware searches of a specific sample.

The API exposes the same operation:

GET /meta/search?mode=grep&q=you're%20absolutely%20right&limit=50

Optional grep parameters:

Parameter Default Behavior
match_case false Set to true to require matching capitalization.
batch_size 128 Maximum candidate records per request, from 1 to 2,048.
time_limited false Set to true to search for about four seconds per request instead. Cannot be combined with batch_size. The web viewer sets it.
eval_set_id Unset Search one eval set instead of the full accessible corpus.
limit 50 Maximum returned matches, from 1 to 200; independent of batch_size.

The response's searched_through is the import time of a record you can read that the request checked; every newer record has been checked. It is null when the request checked no record you can read.

Full-text search pages the same way:

GET /meta/search?q=document.cookie&sort=recency&limit=200

returns up to limit hits and a next_cursor; sort=recency keeps walking older matches, while sort=rank pages through the relevance-ranked candidates of each table's most recent matches. A page with a next_cursor always has rows. Full-text pages show at most three hits per transcript; add diversify=false to get every match, as an export would.

For a whole export in one request, GET /meta/search.csv?q=... (optional eval_set_id) streams every match as text/csv, newest first and without the per-transcript cap, flushing a page at a time. If a later page fails, the last line is #resume-cursor=<cursor>; continue with GET /meta/search from that cursor (sort=recency, diversify=false). hawk search QUERY --csv FILE uses this request and turns that trailer into a --cursor hint.

Both modes also return index_status: complete is false, and incomplete_tables names the tables, while any searched table is still being re-indexed or its index is missing. It is null when the database could not answer within the check's budget, or when the API's database role cannot see every row. The same answer, cached for a minute, is at GET /meta/search/index-status for a scheduled monitor: it requires the usual bearer token and returns 200 with the status, or 503 when the status is unknown.

Pass the response's next_cursor as cursor on the next request, keeping q, match_case, sort and the scope unchanged. Your effective permissions must also remain unchanged; otherwise, start a new search. You can change batch_size, or switch to time_limited, between requests without changing the matching rules. Keep paging even if items is empty, and stop when next_cursor is null. URL-encode both the query and the cursor. mode=fts remains the default; grep always orders newest-first regardless of the sort parameter.

Continuation cursors expire after one hour. If a cursor expires, use Restart search. Retrying a request with the same unexpired cursor is supported. A temporary failure before the first page offers Retry search.

Some Unicode case conversions make the matching text's original position ambiguous. Those results show an unhighlighted excerpt around the match in the lowercased text. Original capitalization is lost in that excerpt; matching results and transcript links are preserved.

Paging is not a snapshot of ongoing imports. Start a new search to include records imported after an earlier page was returned. A scoped search can still return empty pages when records in that scope do not match your phrase or are not visible to you.

When viewing a sample transcript, the viewer's search panel lets you search within that transcript's messages and events.

Two modes are available:

  • Grep (always available): searches the warehouse message_pool and event tables directly — no S3 read. Supports literal, regex, and word-boundary matching. Results include anchored references back to matching messages or events.
  • LLM (subject to model access): runs an Inspect Scout LLM scanner against the transcript. You choose the model to search with (a model is required). The LLM call goes through Hawk's middleman proxy, authenticated with your own credentials — if you don't have access to the chosen model you will receive a 403. The search scope follows the tab you search from: from the messages tab the judge model sees the conversation messages; from the transcript tab it also sees the in-scope events (scores, errors, limits, …) rendered inline as citable [E#] entries. Transcripts of any size can be searched — the scan streams the transcript in bounded memory, splitting it into model-context-sized segments whose results are synthesized into one answer.

Both modes require the same model-group read access as viewing the transcript itself.

Development

To run the viewer locally for development:

cd hawk/www
pnpm install
pnpm dev                  # http://localhost:3000; API defaults to http://localhost:8080

Using a local API server

VITE_API_BASE_URL=http://localhost:8080 pnpm dev

Using a deployed API server

The API sets its session cookies with SameSite=Strict, so a browser on localhost never sends them to a remote API and login loops forever. Proxy the remote API through the dev server so the browser stays same-origin, and use port 5173 (a registered OAuth callback for the default deployment; register http://localhost:5173/oauth/complete yourself if you use your own identity provider). Safari does not accept Secure cookies over plain HTTP on localhost, so use Chrome or Firefox:

API_PROXY_TARGET=https://your-api-server.example.com VITE_API_BASE_URL=/__api pnpm dev --port 5173