Run detail

The run detail page (/ui/runs/{id}) is where you inspect a single run in full: every step’s prompt, output, confidence score, verifier result, and the Trust panel explaining how each gating decision was made. This page covers its four main sections — the run log, live tail, agent trace, and accuracy feedback.

Run log

Each completed run stores a structured event timeline in pipeline_runs.logs. The run detail page shows this as a collapsible log section with timestamped, colour-coded entries (info / warn / error) covering every step start, confidence score, verifier result, skip, escalation, notification, and final status.

Live tail

While a run is in progress the run detail page shows a Live tail panel. It connects via Server-Sent Events (GET /ui/runs/{id}/stream) and streams two categories of events in real-time:

Pipeline log events — fire for all executor types. This isn’t a curated subset: the run log’s event list (the same one in the Run log section above) is a live-publishing list under the hood — every event appended to it streams to the live tail the moment it’s logged, with no filtering in between. In practice that’s 40+ distinct event types covering step start/completion/escalation/abort, verifier and grounding results, fan-out branch lifecycle, budget aborts, retries, loop iterations, and notification delivery — for example step_started, step_completed, step_escalated, verifier_ran, grounding_ran, fan_out_branch_completed, budget_exceeded, step_retrying, notification_sent, run_finished. Watching a live run is watching its real run log build up in real time, not a filtered highlight reel.

Agent trace events — executor: gateway steps only. Each LLM call, thinking block, text response, tool call (with arguments), and tool result streams into the live tail as it happens — not as a batch when the step finishes. These are rendered with compact colour-coded formatting: violet for thinking, cyan for tool calls, green/red for tool results. Content is truncated at 200 chars in the live tail; the full content is in the step detail panel’s Agent trace section.

Late-connecting clients receive a full history replay of everything that happened before they connected, then transition into the live stream — no events are missed.

When the run finishes the page reloads automatically to show the final state. A 5-second polling fallback (GET /runs/{id}) reloads the page if the SSE connection was lost.

Agent trace

Each step’s expanded detail panel includes a collapsible Agent trace section showing the complete internal execution trace: LLM call markers, extended thinking blocks, response text, every tool call with arguments, and every tool result. This is available for executor: gateway steps only — the gateway streams each event back to VectorStep as it fires, which stores the full trace in pipeline_steps.agent_trace.

The trace toggle label shows a count of LLM calls and tool calls at a glance (e.g. 3 LLM calls, 12 tool calls). Tool result content is truncated at 3,000 chars in the stored trace; full content is always available in the gateway’s own logs at DEBUG level.

The same events that populate this panel also appear in the live tail during the step’s execution — the detail panel is the persistent post-run record; the live tail is the real-time view.

For openclaw steps, agent_trace is NULL — OpenClaw does not expose intermediate events to VectorStep.

A skill tool call (load_skill/read_skill_file/run_skill_script) is styled distinctly from an ordinary MCP tool call in both the trace panel and the live tail — labelled “skill” rather than “tool,” showing the skill name (and file/script path, for a file or script call) instead of the raw call arguments, in emerald rather than cyan. An amber ring means at least one call for that skill in this step errored.

Skill badges

A step whose agent has skill grants shows a badge per skill it actually loaded during this run, next to the existing template …/agent … version chips: skill incident-triage@3f2a1c9e0b7d, linking to that skill’s version history. Hovering shows how many times it was loaded, which bundled files were read, and which scripts ran. If the step’s agent was granted a skill it never loaded, a muted line underneath reads granted, not loaded: <names> — the run’s own record of which of an agent’s grants actually did anything, independent of the load-rate averages on Insights — Skills.

Cost

Alongside the token badge, each step and the run total show a cost figure when pricing is configured — 2 decimal places normally, 4 when a step’s cost would otherwise round to a misleadingly-free $0.00. An “unpriced steps: N” note appears whenever some steps couldn’t be priced, so a partial total is never shown as if it were complete.

The badge is colour-coded by which pricing tier priced it, read straight from what was persisted at the time the step ran — never recomputed later, so a historical run’s cost badge shows the same number and the same colour today as it did the day the run happened:

• Blue — a manual pricing.models rate.
• Green — a live OpenRouter rate for the exact model that was actually called (pricing.live_pricing.enabled, provider genuinely openrouter).
• Amber — a live OpenRouter rate borrowed from a different provider’s listing for a similar model (pricing.live_pricing.allow_cross_provider) — a genuine estimate, not the rate for the API that was actually called.

A step whose primary/verifier/grounding-judge calls resolved to different tiers shows a ✻ marker next to the amount and is coloured by its weakest tier — a badge never looks more certain than its shakiest input. Hover any cost badge for a full breakdown: which component contributed what, from which tier, and (for a live tier) the exact OpenRouter catalog id and per-Mtok rates used. See Cost accounting for the full model, including how each tier decides whether it counts toward budget.max_usd.

Accuracy feedback

After a run completes, any user can mark it with a human judgement of whether the pipeline’s outcome was correct. The feedback widget appears at the bottom of every finished run’s detail page (hidden for running and interrupted runs).

Outcomes:

Outcome	When to use
Correct	The pipeline did what it was supposed to do
Partial	The pipeline did useful work but didn’t fully achieve the goal
Incorrect	The outcome was wrong or misleading

An optional notes field lets you record why — useful context when reviewing patterns later. Submitting again overwrites the previous outcome (upsert).

Bulk grading from the runs list

When you have many runs to grade with the same outcome, you don’t need to open each one. On /ui/runs (as an operator or admin), tick the finished runs you want — running and interrupted runs have no checkbox — or use the header checkbox to select every run on the current page. A bar shows “N selected on this page” with an optional notes field and Correct / Partial / Incorrect buttons. The list reloads with a summary (“Marked 12 run(s) correct (10 new, 2 updated)”).

• Select-all applies to the current page only (page size follows limit). There is no “select everything matching the filter” yet.
• It is the same upsert as the single widget: a run that already has a grade is updated. Runs that no longer exist are reported as not_found and don’t fail the batch.
• A bulk call is capped at feedback.bulk_max items (default 500).
• Bulk marks count exactly like single marks toward accuracy, calibration and promotion readiness, so use it for runs you have actually reviewed. Step-level grading is not offered from the runs list; it lives on the marking queue (and the API / MCP tool).
• One feedback.bulk_submit row is written to the audit log per bulk call (see Security). The single-run widget does not write an audit row.

Where accuracy data surfaces:

• Run detail — the feedback widget; shows current outcome if already marked.
• Pipeline detail — a colour-coded correct/partial/incorrect bar with counts, and a “View breakdown →” link.
• Insights — Overview — an “Accuracy” stat card showing the % correct of all marked runs in the selected time range.
• Pipeline accuracy page (/ui/pipelines/{name}/feedback) — the full breakdown:
  • Summary cards (total marked, correct, partial, incorrect with percentages)
  • Overall accuracy distribution bar
  • Accuracy by configuration table — runs are grouped by a fingerprint of the exact (step sequence × agents × models) combination. When you change a model, add a step, or swap an agent, the new runs fall into a new group automatically, so you can directly compare accuracy before and after any pipeline change without manually tagging versions.
  • Chronological table of every marked run with its outcome, run status, config fingerprint, and notes.

Per-step feedback

In addition to run-level feedback, you can mark an individual step execution correct/partial/incorrect. The control appears inside each finished step’s expanded detail panel (same collapsible body as the parsed output and agent trace), so marking is optional and sparse — mark only the step(s) you have an opinion on. Fan-out branches (triage/0, triage/1, …) are marked independently, since each is its own step execution.

• Steps Insights (/ui/insights/steps) — the pipeline/agent/model breakdown table has an Accuracy column, and the per-step drilldown has an Accuracy mini-card, both computed as correct / total_marked over the selected time range, production-scoped.
• vectorstep_step_feedback_total exposes the same counts for Grafana/alerting.

Per-step feedback does not affect this run’s own gating or flow control — marking a step doesn’t change what already happened. It does feed calibration: a human mark on a step execution is the highest-precedence label calibration uses when bucketing that (step, agent, model, provider, prompt_hash, agent_version) configuration’s track record.