REST API

The service exposes a full JSON API: run triggering and inspection, live SSE run tailing, pipeline/step CRUD with validation, per-pipeline and per-step analytics, calibration bins, prompt/agent version history, readiness readouts, accuracy feedback, Prometheus metrics, and health probes.

Every route below except GET /health/GET /favicon.svg/GET /metrics (open by default) requires a bearer token — each section names the role it needs. See Security for the token/role model.

This page covers the core management endpoints — triggering and inspecting runs, feedback, pipeline listing, metrics, and health. For the read-only pipeline/step/agent analytics endpoints, see Analytics API. For the endpoints that create, update, validate, and delete pipelines and steps, see Write API — both were added to back the VectorStep Service MCP.

Triggering runs

POST /webhook?source=<source>

Role: webhook

Trigger a run. Returns immediately — the pipeline runs in the background.

• → {"status": "accepted", "run_id": "<uuid>"}
• → {"status": "deduplicated", "run_id": "<uuid>", "reason": "..."} — see Webhooks for idempotency and deduplication.
• → {"status": "skipped_testing", "pipeline": "...", "reason": "..."} — see Pipeline stages; pass ?allow_testing=true to run a stage: testing pipeline from this source anyway.

POST /pipelines/{name}/run

Role: operator

Manually trigger a pipeline by name — powers the UI’s Run now button.

Note

Gated at operator rather than webhook — this is an internal/management action, at the same trust boundary as /reload’s admin gate but one step down, not the public ingestion path that a webhook-role token authenticates. Runs triggered here are unattributed (team=None) regardless of who’s authenticated, since a session/token here carries no team the way a webhook-role token does.

• body: same shape as a generic webhook payload (see Webhooks), pipeline forced from the path
• → {"status": "accepted", "run_id": "<uuid>", ...}

Reload

POST /reload

Role: admin

Reload the step library and all pipeline YAMLs from disk without restarting.

→ {"status": "reloaded", "pipelines_loaded": 3}

The Gateway has its own POST /reload (agents and skills) — the two reload independently. On Kubernetes, see Changing config on Kubernetes.

SIGHUP also triggers a reload:

kill -HUP <uvicorn-pid>

Schedules

GET /schedules

Role: viewer

List active cron schedules.

→ {"schedules": [{"pipeline": "...", "cron": "...", "next_run": "..."}]}

Security policies

GET /security-policies

Role: viewer

List loaded security policies — name, description, default, and rule count per policy. Never the rules themselves.

→ {"security_policies": [{"name": "...", "description": "...", "default": "allow", "rule_count": 2}, ...]}

GET /security-policies/{name}

Role: viewer

Full rule list for one policy — see Security policies. 404 if the name isn’t in the loaded registry.

→ {"config": {"name": "...", "description": "...", "default": "allow", "rules": [...]}}

No POST/PUT/DELETE exists for this prefix at any token tier — policy content is config.yaml/filesystem-only, deliberately never reachable through the pipeline-write trust tier. See Security policies.

GET /pipelines/{name} also gains a security_policy_status field: for each policy the pipeline declares, whether it currently resolves (exists in the loaded registry) and passes.

Runs

GET /runs

Role: viewer

List runs, newest first.

Filters: ?status=escalated, ?pipeline=alert-triage-critical, ?team=payments, ?stage=production|testing.

Pagination: ?limit=50&offset=0 (max 200).

→ {"runs": [{id, pipeline_name, source, status, team, stage, triggered_at, completed_at}, ...]}

stage is "testing" or "production" (see Pipeline stages), included on both this list and the full detail response below — it’s persisted per-run, so it reflects the pipeline’s stage at the time the run was triggered rather than its current config.

GET /runs/{run_id}

Role: viewer

Full run detail — includes all steps with confidence scores and parsed output.

POST /runs/{run_id}/rerun

Role: operator

Re-run a pipeline from a specific step.

• body: {"from_step": "step-name"}

Feedback

POST /runs/{run_id}/feedback

Role: operator

Submit or update human accuracy feedback for a run (outcome: correct | partial | incorrect).

• body: {"outcome": "correct", "notes": "..."}
• → {"run_id": "...", "outcome": "correct", "notes": "...", "submitted_at": "..."}

Upserts — submitting again overwrites the previous outcome and notes.

GET /runs/{run_id}/feedback

Role: viewer

Get current feedback for a run.

→ {"feedback": {run_id, outcome, notes, submitted_at}} or {"feedback": null}

POST /runs/{run_id}/steps/{step_name}/feedback

Role: operator

Submit or update human accuracy feedback for a single step execution (outcome: correct | partial | incorrect). step_name may contain / for fan-out branches (e.g. triage/0) — the route uses a path converter to match it.

• body: {"outcome": "correct", "notes": "..."}
• → {"run_id": "...", "step_name": "...", "outcome": "correct", "notes": "...", "submitted_at": "..."}

Upserts — submitting again overwrites the previous outcome and notes.

GET /runs/{run_id}/steps/{step_name}/feedback

Role: viewer

Get current feedback for a single step execution.

→ {"feedback": {run_id, step_name, outcome, notes, submitted_at}} or {"feedback": null}

POST /feedback/runs/bulk

Role: operator

Apply one accuracy grade to many runs. Same upsert as POST /runs/{run_id}/feedback.

• body: {"outcome": "correct", "notes": "...", "run_ids": ["...", "..."]} (notes optional)
• → {"results": [{"run_id": "...", "status": "created|updated|not_found"}], "created": n, "updated": n, "not_found": n}

Unknown run ids are reported per item as not_found and do not fail the request. Duplicate ids are collapsed. More than feedback.bulk_max ids (default 500) returns 422; a bad outcome or empty list returns 400. One feedback.bulk_submit audit row is written per call. (The prefix is /feedback/… rather than /runs/feedback/… on purpose: the latter would be shadowed by the /runs/{run_id}/… routes.)

POST /feedback/steps/bulk

Role: operator

Same, for step executions.

• body: {"outcome": "partial", "notes": "...", "items": [{"run_id": "...", "step_name": "..."}]}
• → as above, each result also carrying step_name

The UI for this is the marking queue.

Approvals

Pending human-approval gates (executor: human, see Human in the loop).

GET /approvals

Role: operator (not viewer — the listed tokens are bearer credentials for approving)

→ {"approvals": [{"token", "pipeline", "step", "team", "run_id", "created_at", "message"}]}, newest first.

POST /approvals/bulk

Role: operator

• body: {"action": "approve" | "reject", "tokens": ["...", "..."]}
• → {"results": [{"token": "...", "status": "resolved|not_found_or_already_resolved"}], "resolved": n, "skipped": m}

Each resolved token gets its own approval.approve / approval.reject audit row with the caller’s identity. Unknown and already-decided tokens are reported as not_found_or_already_resolved (one status, since they can’t be told apart once a decision is made) and are not errors. More than approvals.bulk_max tokens (default 200) returns 422.

Golden sets

Golden sets themselves are curated through /ui/golden-sets/* form routes (see UI overview and Replay / shadow evaluation — Scheduled regression replay) — these two are the only JSON endpoints this feature area has, added for synthetic members and the VectorStep Service MCP.

GET /golden-sets/{golden_set_id}/members

Role: viewer

Every member of a golden set — recorded and synthetic — as JSON.

→ {"golden_set_id": "...", "members": [{member_id, member_type, step_id, pinned_label, pinned_source, added_at, source_run_id, synthetic_input, synthetic_expected_output}, ...]}

member_type is "recorded" or "synthetic". For a "recorded" member, synthetic_input/synthetic_expected_output are null and source_run_id names the run its pinned step belongs to (null if that step has since been deleted — see Replay & scheduled regression). For a "synthetic" member, step_id/source_run_id are null and synthetic_input/synthetic_expected_output carry what was authored.

POST /golden-sets/{golden_set_id}/synthetic-members

Role: operator

Author a golden-set member from scratch — an input/expected-output pair with no real pipeline_steps row behind it — rather than pinning an existing one.

• body: {"input": {"severity": "critical"}, "expected_output": "escalate to on-call"}
• → 201 {"member_id": "...", "golden_set_id": "...", "member_type": "synthetic"}

input’s keys must be a subset of the golden set’s step’s pipeline’s context_template.include fields — an unrecognized field 422s, naming it. expected_output must be non-empty. See Synthetic members for the single-step scope limit and why this is always treated as the “correct” reference.

Live tailing

GET /ui/runs/{run_id}/stream

Role: viewer (401s rather than redirecting when missing — see Security)

Server-Sent Events stream for live run tailing. See UI: run detail.

Pipelines and observability

GET /pipelines

Role: viewer

List loaded pipelines — includes stage and tags alongside name/description/version.

GET /metrics

Role: none by default; viewer when auth.metrics_auth: true

Prometheus metrics — runs/steps by status, step duration histograms, verifier veto rate.

GET /health

Role: none

Liveness/readiness probe — also surfaces in-process concurrency state.

→ {"status": "ok", "version": "...", "active_runs": 0, "max_concurrent_runs": 10, "config_errors": 0}

config_errors is the number of pipeline or library-step files the service skipped at startup because they failed to load (see Changing config on Kubernetes → a bad file). It’s a count only — which files, and why, is in the log and in a banner on every UI page — and status stays ok so probes keep passing. Alert on it being non-zero if a skipped pipeline would matter to you.