Production Prompt Orchestration
VectorStep turns webhooks into multi-step AI pipelines. You decide what a step has to prove - a second opinion, evidence checks, deterministic facts, a track record - before it acts on its own.
⤢ Click to enlargeEvery pipeline’s steps, agents and run history in one page — the structure, not just the outcome
An agent finishes a task and says “I’m 95% confident.”
Should you believe it?
LLMs can be completely wrong and still sound calm, structured, and sure of themselves. Confidence, as an agent reports it, is a style of writing as much as a measurement. VectorStep doesn't have to rely on one number - it can build a trust vector from independent signals and gate an autonomous action on the weakest one. Not an average. The floor.
What you get on day one
Every gating signal on this page - verifier, grounding, calibration, readiness - is additive. None of it is required to run a production pipeline.
Triggers, steps and flow control are declarative config, versioned like everything else you own. No UI required to author or ship one.
Fan out over discovered work at runtime, run branches concurrently, and merge the results back into the next step.
Conditional steps, a per-step failure policy, and automatic retries - because real pipelines don't get to assume every step succeeds.
Rerun a past run from any step to debug it, or replay recent production runs against a candidate model or prompt before you promote it.
Route an approval to Telegram, Slack or Teams and the pipeline waits - no separate approvals system to bolt on.
Every step's spend is priced per model and rolled up per pipeline, team and provider — with budget caps to stop a run before it gets expensive.
The reusable step library is why a real pipeline config (see it below ↓) stays a dozen lines instead of a hundred. Write the prompt template and the output contract once, then reference it withuse: sre-investigationfrom any pipeline.
It's more than a shortcut: every pipeline that uses a library step feeds its accuracy and calibration data back to it - tracked per model and prompt version - so the step your team shares gets more provably trustworthy the more it's relied on, not just shorter to write.
⤢ Click to enlargeDeveloper experience
Every pipeline and every step is YAML in version control. Changes go through the same code review as the rest of your codebase.
POST /reloador SIGHUPpicks up config changes without dropping a run in flight.
A dedicated validate endpoint checks a pipeline against the schema and never mutates state - wire it into CI before anything merges.
SQLite for zero-infra local development, Postgres for production - no branching logic, no surprises when you promote.
Prometheus metrics and OpenTelemetry tracing ship with the service - plug in your existing stack, nothing to bolt on.
Two MCP servers expose pipeline and agent authoring directly to your coding assistant — Claude Code, Claude Desktop, or any other MCP-compatible client - write, inspect and iterate on pipelines without leaving it.
The trust vector
What the agent says about its own work. The starting point - and the least trustworthy number in the whole system, treated accordingly.
A second agent reviews the reasoning - or independently redoes the task, blind. A verifier can lower confidence, never raise it. Consensus is not comfort.
Is every LLM claim backed by an actual tool call in the agent’s own trace? Plausible-sounding prose doesn’t count. Evidence does.
Real commands, real queries, real approvals - no model in the loop. One failed hard check forces trust to zero. No averaging. No partial credit.
Threshold 75% - the agent was “95% confident” the whole time.
Every step of this chain — with the exact numbers for that specific run — is visible in the run’s Trust panel under “How was this calculated?”. Nothing is a black box.
Calibration
VectorStep bins every marked outcome per agent, model, provider, prompt version and agent version, then measures what each confidence band was actually worth. If an agent says 90% but its history at that level is 67% - you’ll see it, in plain English, before it ever bites you.
sre-agent · claude-sonnet · confidence 90–100%
31 marked runs in this band
Config-driven
No SDK, no DSL, no redeploy. Pipelines, reusable steps, verifiers, grounding, calibration and hard checks are declarative config - versioned in git, hot-reloaded on change, validated before a byte is written.
name: alert-triage
description: Triage critical alerts and notify on completion
trigger:
source: alertmanager
match: { severity: critical }
steps:
- name: investigate
use: sre-investigation # reusable step library
executor: gateway
executor_config:
agent: sre-agent
confidence_threshold: 0.75
on_low_confidence: escalate # a human sees it insteadNo verifier, no grounding, no calibration, no readiness - a legitimate way to run this pipeline in production.
name: alert-triage-critical
description: Triage critical alerts, act only when trust survives every check
trigger:
source: alertmanager
match: { severity: critical }
steps:
- name: investigate
use: sre-investigation # reusable step library
executor: gateway
agent: sre-agent
confidence_threshold: 0.75
on_low_confidence: escalate # a human sees it instead
verifier:
mode: independent # blind second agent
combination_strategy: minimum
grounding:
enforce: true # claims need evidence
calibration:
enforce: true # trust the track record
on_uncalibrated: escalate
deterministic_checks:
- type: shell # still breaching, live?
command: ./checks/alert-still-firing.shThe whole safety story is YAML you can read in review, too.
Gated is Simple plus four opt-in blocks - verifier, grounding, calibration, deterministic checks. Nothing in Simple is removed to get there.
Operate with evidence
Per-step breakdowns by agent, model and provider - success rate, tokens, latency and judged accuracy, un-blended. “Did that prompt edit help?” has a diff and a number.
⤢ Click to enlargeOwner-defined readiness criteria across four tiers - operational, confidence, accuracy, calibration - judged against real evidence, with a guided builder that previews any criteria change in ~300ms.
⤢ Click to enlargeA marking queue of every step lacking human accuracy feedback - the exact labels calibration and readiness are built from. Live-tail any run while it happens.
⤢ Click to enlargeFrom first webhook to a gated, calibrated production pipeline - the quick start takes about ten minutes.