Insights

The Insights section (/ui/insights/*) is a set of nine pages giving cross-pipeline rollups over runs, steps, agents, models, providers, MCP tool usage, skill usage, and teams. Every Insights page defaults to stage: production runs only, consistent with every other rollup surface in the UI — but each one also carries a Stage selector (next to the existing time-range selector) that switches the whole page to testing on request. Nothing changes unless you explicitly ask: leave it alone and every total, chart, and drilldown behaves exactly as it always has.

There is deliberately no “all stages” option. Every number on these pages — success rate, accuracy, calibration — is a rate or a track record, and blending production and testing runs into one figure would produce a number that doesn’t correspond to either population’s real behaviour. Pick the one population you actually want to reason about; ?stage=all returns a clear 422 rather than silently mixing the two.

Reach for testing when a pipeline you’re actively developing (still stage: testing) has real accumulated history you want to look at in aggregate, rather than one run at a time in the run-detail page — the same reasoning behind /ui/runs’s own ?stage= filter, extended here to every rollup, not just the raw run list. Month-to-date team budget spend (on the Teams page) is the one figure that stays production-only regardless of this selector — a real budget shouldn’t count exploratory testing-stage runs against it.

Every “cost” total on every page below carries a per-source breakdown — how much of that figure came from manually-priced steps versus live OpenRouter pricing (exact or cross-provider) — since cost is persisted as three separate tiers. See Cost accounting for what each tier means.

Insights — Overview

/ui/insights shows run/failure/token/accuracy totals, runs by team, and MCP tool-use counts, over a selectable time range (24h/7d/30d/all-time). This is the landing page for the section — a single screen of the numbers you’d otherwise have to piece together from the Runs and Pipelines pages.

When pricing.live_pricing.enabled is set (see Cost accounting), this page also shows a Live reference pricing panel — OpenRouter’s current list price for every model in use that could be fuzzy-matched, clearly disclaimed as approximate and not necessarily what you actually pay.

Insights — Pipelines

/ui/insights/pipelines shows per-pipeline run/failure/duration/token/cost totals and a top-pipelines table. Drilling into a pipeline gives a status/accuracy breakdown, a timeseries, recent runs, and a step/agent/model breakdown table for that pipeline specifically.

Insights — Steps

/ui/insights/steps shows per-step run/failure/duration/token totals and a top-steps table. The per-step drilldown gives a status breakdown, a timeseries, recent executions, and a pipeline/agent/model breakdown table — this is also where calibration bins for every agent/model/prompt-version combination live (see How confidence and calibration work).

Calibration is the one number on this page where the Stage selector matters more than anywhere else, and this page says so explicitly with a banner above the tables: viewing Production shows exactly the bucketed data an enforced calibration: {enforce: true} gate evaluates against by default — same bin width, same minimum-sample threshold, same population. Viewing Testing shows a step’s testing-stage track record instead, a genuinely separate, parallel view (never blended with production on this page) that’s useful for building confidence in a step before promoting it — but a live gate ignores it by default, no matter what’s selected here. If you’ve turned on an enforced calibration gate and want to sanity-check the number it’s actually using, this page has to be on Production to show you that number — unless that step also sets calibration.include_testing: true, in which case its own gate blends both stages and neither single-stage view here is its exact number; check that step’s run detail page instead.

Insights — Agents

/ui/insights/agents shows per-agent step/success-rate/duration/token totals and a top-agents table. The per-agent drilldown gives a status breakdown, a timeseries, recent executions, and a pipeline/step/model breakdown table.

Insights — Models

/ui/insights/models shows per-model (provider-qualified — see Agent Library — model display) success-rate/duration/token/cost totals and a top-models table, with a per-model drilldown (status breakdown, timeseries, recent calls, and a pipeline/step/agent breakdown table). This page is executor: gateway steps only — it has no data for OpenClaw-executed steps, since only the gateway executor records a model per step.

Insights — Providers

/ui/insights/providers groups calls/success-rate/duration/token/cost totals by LLM provider (anthropic, openrouter, azure, etc.), with a top-providers table and a per-provider drilldown of the same shape as the other Insights pages.

/ui/providers redirects here. This page also has one piece of special-casing not found on any other Insights page: it falls back to a best-effort provider guess parsed from the model string for any row with no provider value recorded, since the entire point of this page is bucketing by provider, and a page that couldn’t bucket those rows at all would undercount them. Every other Insights page instead leaves an unrecorded provider as a bare model name rather than guessing — this page is the deliberate exception, because guessing here is strictly better than an artificial gap in the provider totals.

Like Models, this page is executor: gateway steps only.

Insights — MCP

/ui/insights/mcp shows tool call usage extracted from the agent trace on executor: gateway steps — calls/errors by tool and by server, and a per-tool drilldown showing which pipelines/steps/agents call it, over a selectable time range. OpenClaw steps don’t expose intermediate events, so they contribute nothing here. This page is analytics on tool usage; for the live tool/server registry (schemas, running/pid/restart_count), see the MCP Tools page (/ui/mcp).

Skill tool calls (load_skill/read_skill_file/run_skill_script) are not counted on this page — they moved to Insights — Skills below, since a skill isn’t an MCP server and deserves its own load-rate framing rather than being folded into a generic tool-calls bucket.

Insights — Skills

/ui/insights/skills shows Gateway skill usage across every agent granted each skill — headline cards (skills used, total loads, overall load rate, script runs, script error rate), a loads-over-time chart, and a table of skills with load rate (loaded steps ÷ granted steps — the single most useful number for judging whether a skill’s description actually gets it triggered), loads, script runs, and script errors.

A “Granted but never loaded” panel lists any skill with granted_steps > 0 and loaded_steps == 0 in the selected range — a signal the skill’s one-line description may not be triggering the model to load it; see Writing good skills.

Clicking a skill row opens a drilldown: its own loads-over-time chart, a loaded vs. not loaded comparison (failure rate and average effective confidence for steps that loaded the skill versus steps that were granted it but didn’t — labelled correlation, not causation, since agents may reach for a skill specifically on harder inputs), recent runs that loaded it, and a pipeline/step/agent breakdown. See Skills for what a skill is, and the Skill Library below for the per-skill browsing page this links out to.

Insights — Teams

/ui/insights/teams shows per-team run/success-rate/duration/token/cost totals and a top-teams table. The per-team drilldown gives a complete picture of what a team uses and where — pipelines used, and a pipeline/step/agent/model breakdown table — plus its token/cost spend, for informed cost decisions. A NULL team is bucketed as “Unattributed” rather than dropped.

If pricing.team_budgets is configured, this page also shows a month-to-date budgets section — a spend-vs-budget bar per team, advisory only (see Cost accounting).

Agent Library

The /ui/agents page provides a unified library of agents across all configured executor backends. Agents are fetched live from each backend and merged into a single list with executor badges.

Agents are uniquely identified by executor:name — e.g. openclaw:sre-investigation and gateway:sre-investigation are treated as distinct agents. This prefix is stored in pipeline_steps.agent so run history, success rates, and model usage are attributed correctly per backend.

Executor	Agent list	Agent files
openclaw	OpenClaw Gateway WS — agents.list RPC	agents.files.get RPC — SOUL.md, TOOLS.md, IDENTITY.md tabs
gateway	VectorStep Gateway REST — GET /agents	GET /agents/{name}/soul (Soul tab) · GET /agents/{name}/agent (Config tab — raw agent.yaml)

Both backends are queried concurrently. If one is unreachable, the other’s agents still show with a warning banner. If both fail, stub entries from DB run history are surfaced.

The Config tab on a gateway agent detail page shows the raw agent.yaml — model, max_tokens, and the list of MCP tool names the agent has access to.

Overview tab — a per-model breakdown table (runs, succeeded, failed, success rate, avg duration, avg tokens in/out, last run), two “usage over time” line charts (runs and tokens, both split by model), and a recent activity list of the last 15 steps this agent ran across any pipeline — each row links to its pipeline and its run detail page.

Steps tab — which pipeline steps this agent executes, broken down by pipeline and model (runs, success rate, avg tokens, last run). The same step name can be wired to a different model in different pipelines, so pipeline is a first-class column here rather than folded away.

All of the above is scoped to stage=production runs, same as every other rollup surface.

Skill Library

The /ui/skills page is a read-only browser for skills loaded on the VectorStep Gateway (SKILL.md + bundled files an agent can load on demand at runtime — see Skills). Skills are authored as files and pushed into the Gateway’s skills_dir by an engineer; there is no create/edit/delete here, only browsing, usage, and version history.

The list table shows, per skill: name, description (the field that decides whether an agent’s model actually loads it — shown in full, expandable), version, bundled file/script counts, which agents are granted it (chips linking to their agent detail page), and 30-day loads/load rate/last-used, fetched from the Gateway’s GET /skills and GET /agents concurrently. A warning panel appears above the table if any agent grants a skill name the Gateway doesn’t currently have loaded.

Clicking a skill opens its detail page, with up to five tabs:

• Overview — the rendered SKILL.md body.
• Files — every bundled file, with an “executable” badge for one allowlisted via the Gateway’s security.skills.allowed (allowlisting is config-only — this page can never grant it). Clicking a file shows its content, Markdown rendered, anything else as plain text, and a binary file as “not shown.”
• Agents — every agent currently granted this skill, with its live agent_version.
• Usage — the same load-rate/script-activity numbers as Insights — Skills, scoped to just this skill, with its own production/testing stage toggle and a link out to the full Insights drilldown.
• Versions — VectorStep’s own content-hash snapshot history for this skill (SKILL.md + bundled files at each version VectorStep observed), with a diff between consecutive versions. This is the only place a skill’s previous content survives once its files change — it works the same way agent version history does (see How confidence and calibration work).

If a skill is deleted from the Gateway but VectorStep still has snapshots for it, the detail page renders from history alone (Usage and Versions tabs only) with a banner explaining it’s no longer loaded — history outlives the files.

Model display and the provider column

Wherever a model name is shown alongside run history (this page, /ui/steps, and the Insights pages), it’s prefixed with its provider when the DB actually recorded one — e.g. anthropic/claude-sonnet-5, openrouter/deepseek/deepseek-v4-pro. pipeline_steps.provider is only populated for executor: gateway steps (from the Gateway’s agentMeta.provider); other executors leave it NULL and the bare model name is shown as-is — the UI does not guess a provider it has no evidence for, since a wrong guess is worse than no answer. (The one deliberate exception to that policy is the Insights — Providers page above.)