Tutorial: build your first agent

The quick start got a pipeline running end to end, but it borrowed a ready-made agent to do it. This tutorial builds one from nothing: two real MCP servers, an agent.yaml and a soul.md you write yourself, and a prompt that puts both tools to work.

It deliberately turns none of the trust knobs on. That’s the next tutorial — this one is just about getting an agent to do something real and watching it happen.

What you’ll build

A first-responder agent that triages the same critical alert from the quick start, but actually gathers evidence before handing off:

1. Checks GitHub’s public status API — if the alerting service depends on GitHub (CI, package registry, container registry), an upstream incident changes the whole story.
2. Checks a local known-issues.md file — has this exact alert already been triaged and understood?

Two tools, two MCP servers, zero accounts to sign up for.

Models used for testing

This series was validated internally against openrouter/anthropic/claude-sonnet-4.5 for first-responder and openrouter/anthropic/claude-haiku-4.5 for every other agent introduced later in the series (verifiers, grounding-judge, upstream-checker, generic-pipeline-step). Other models and providers should work too, but exact wording and confidence numbers can vary by model — worth knowing if what you see doesn’t match a “what you should see” section exactly.

Prerequisites

• The quick start completed — Gateway and VectorStep both running.

• npx and uvx on your $PATH — the Gateway spawns both MCP servers below as subprocesses using these. Install whichever you don’t already have:

  # uvx ships with uv:
  curl -LsSf https://astral.sh/uv/install.sh | sh
  
  # npx ships with Node.js — macOS:
  brew install node
  # npx ships with Node.js — Debian/Ubuntu:
  sudo apt install nodejs npm
  # otherwise see https://nodejs.org/

  Verify both before continuing — if either command isn’t found, the install above didn’t put it on $PATH (a fresh shell often fixes this):

  uvx --version
  npx --version

1. Give the filesystem server something to read

MCP’s filesystem server needs a real directory to scope itself to. Create one and drop a known-issues log in it:

mkdir -p ~/vectorstep-tutorial
cat > ~/vectorstep-tutorial/known-issues.md <<'EOF'
# Known issues

- payments-api: intermittent 5xx during the nightly batch export job
  (02:00-02:15 UTC). Not a page — self-resolves within minutes.
EOF

echo ~/vectorstep-tutorial   # note this absolute path — you need it below

2. Add both MCP servers to the Gateway

In the Gateway’s ~/.vectorstep-gateway/config.yaml, add — using the absolute path from above in place of /absolute/path/to/vectorstep-tutorial:

mcp_servers:
  fetch:
    command: uvx
    args: ["mcp-server-fetch", "--ignore-robots-txt"]
  filesystem:
    command: npx
    args: ["-y", "@modelcontextprotocol/server-filesystem", "/absolute/path/to/vectorstep-tutorial"]

The filesystem server’s argument must be an absolute path — it scopes every file operation to that directory and everything under it, and won’t expand ~ itself.

--ignore-robots-txt is needed because githubstatus.com/robots.txt disallows /api/ for automated fetchers — a courtesy convention aimed at crawlers, not an access control, but mcp-server-fetch honours it by default and refuses the request otherwise. This deliberately overrides that site’s stated preference for this one tool; it’s not something to apply automatically to every fetch server you configure later.

3. Write the agent

Agents live under the Gateway’s agents_dir (default ./agents/, relative to ~/.vectorstep-gateway/). The agent below is written narrow on purpose — one job, two tools, an explicit output contract — following the principles in Writing good agents, worth a read once this tutorial is done.

cd ~/.vectorstep-gateway
mkdir -p agents/first-responder

~/.vectorstep-gateway/agents/first-responder/agent.yaml:

name: first-responder
model: anthropic/claude-sonnet-4-6
max_tokens: 4096
tools:
  - fetch
  - filesystem

~/.vectorstep-gateway/agents/first-responder/soul.md:

# First Responder

You are a first-response triage agent for infrastructure alerts. Your job is
narrow: gather two pieces of evidence and hand off a clear brief. You do not
remediate anything, and you do not guess at a root cause you haven't checked
for.

## What you do

1. Use the `fetch` tool to check whether GitHub itself is having an
   incident — relevant if the alerting service depends on it (CI, package
   registry, container registry).
2. Use the `filesystem` tool to read `known-issues.md` and check whether
   this exact alert has already been triaged before.
3. Summarise what you found and hand off.

## Confidence

Confidence measures how completely you gathered the two pieces of evidence
above — not how serious the alert is. Both tool calls succeeded and gave you
a clear answer → confidence should be high. A tool failed, timed out, or gave
you nothing useful → say so honestly and score low, rather than filling the
gap with a plausible-sounding guess.

## Output format

Respond with ONLY the JSON object your prompt asks for. No preamble, no
markdown fences, no commentary outside the JSON.

Restart the Gateway so it picks up the new mcp_servers entries and the new agent — hot reload covers agent config changes, not new MCP server subprocesses, so a restart is the safe move here.

If you just installed uv/Node.js as part of the prerequisites above, restart the Gateway from a new terminal window, not the one it’s already running in. The Gateway spawns uvx/npx by resolving them against its own process environment, captured when it started — an installer updating your shell profile doesn’t reach a process that’s already running, so restarting it in that same old shell still fails with [Errno 2] No such file or directory even though uvx --version works fine when you type it yourself. A fresh terminal (or source-ing your shell profile first) picks up the updated PATH.

4. Check the tools actually loaded

curl http://localhost:18780/mcp/tools

You should see tool entries under both fetch and filesystem. If a server is missing, check the Gateway’s startup logs — a bad command/args fails loudly there.

5. Wire the pipeline to your new agent

Switch back to the service side, in ~/.vectorstep/service/ — the quick start’s ~/.vectorstep/service/pipelines/alert-triage.yaml already triggers on severity: critical and pulls its step from the step library (use: first-line-triage). Rather than add a second pipeline that would collide with the same trigger match — pipeline resolution is first-match-wins, so the two would race — replace that file’s contents to point at your new agent with an inline step instead of the library one:

name: alert-triage
description: First-responder agent gathers evidence before anyone escalates
trigger:
  match: { source: alertmanager, severity: critical }
  dedup:
    enabled: false   # quick-start convenience — see the quick start's note on this

context_template:
  include:
    - severity
    - summary

steps:
  - name: triage
    executor: gateway
    executor_config:
      agent: first-responder
      session_key: "agent:first-responder:{{pipeline_run_id}}:triage"
    prompt_template: |
      A {{severity}} alert fired for {{labels.service}} in {{labels.environment}}.
      Summary: {{summary}}

      1. Use the fetch tool to check https://www.githubstatus.com/api/v2/summary.json
         for any active GitHub incident.
      2. Use the filesystem tool to read known-issues.md and check whether an
         entry already matches this alert.

      Return ONLY this JSON, no other text:
      {
        "confidence": 0.0,
        "summary": "One sentence: what's happening and what you found",
        "next_step_context": "",
        "upstream_incident": true,
        "known_issue": true,
        "reasoning": {
          "supports": "Evidence that makes this alert credible",
          "contradicts": "Evidence that suggests noise or a known cause",
          "assumptions": "What you're assuming in the absence of data"
        }
      }

confidence, summary and next_step_context are the three mandatory fields every agent response must include — see the LLMOutput contract — next_step_context can be an empty string for a terminal step like this one, but it has to be present or the response fails validation. Everything else in the JSON above (upstream_incident, known_issue) is a free-form extra field, stored and available to any later step as {{steps.triage.upstream_incident}}.

Notice what’s not here: no confidence_threshold, no on_low_confidence, no verifier. This is rung 0 on the trust ladder — a fully working pipeline with no gating at all, which is a legitimate place to stop for a step that only informs rather than acts.

Reload the service — run this, and the trigger command below, from ~/.vectorstep/service (the directory quick-start’s step 2 left you in):

curl -X POST http://localhost:8000/reload

6. Trigger it

Reuse the same fixture from the quick start — the point of this tutorial is the agent, not a new trigger. This pipeline is still stage: testing by default, same as in the quick start, so allow_testing=true is required:

curl -X POST "http://localhost:8000/webhook?source=alertmanager&allow_testing=true" \
  -H "Content-Type: application/json" \
  -d @tests/fixtures/alertmanager_critical.json

7. Watch it run

Open the run in http://localhost:8000/ui/ and look at the step’s trace. You should see two real tool calls — fetch hitting GitHub’s status API and filesystem reading known-issues.md — plus the agent’s JSON response built from what they returned, not from what the model assumed.

What you should see

Expect the triage step to show completed at (or near) 100% confidence — a real contrast with the quick start’s escalated result at ~10%. That’s the whole point of this tutorial: same alert, same shape of task, but this agent actually has evidence to reason from instead of bare labels.

The summary should reference both tools concretely — something like “GitHub reports all systems operational and known-issues.md documents intermittent 5xx during the nightly batch export job” — and next-step-context should reflect the known-issue match, e.g. checking whether the alert falls inside that documented window. In REASONING → CONTRADICTS, you should see the model explicitly weighing the known-issue match against the alert firing at all — evidence of it actually reasoning over what the tools returned, not pattern-matching a generic “alert fired, escalate” response. The two extra fields from the prompt’s JSON schema, upstream_incident and known_issue, show up under OTHER FIELDS.

Nothing here is being checked yet — there’s no confidence threshold, no verifier, no grounding enforcement on this step (that’s deliberate, per “what’s not here” above). A confident-sounding response and a confident-and-grounded one look identical until something actually verifies the trace behind it. That’s exactly what grounding does next: not whether the output sounds right, but whether it’s backed by a real tool call in the agent’s own trace.

Where next

Go to Turn on the trust knobs next: it takes this exact pipeline and adds a confidence floor and a verifier — the natural continuation of the “nothing here is being checked yet” point above.

Once you’re comfortable with the mechanics:

• Writing good agents — the principles behind the agent you just wrote: narrow scope, minimal tools, honest uncertainty, and why each of those matters more than it looks like it should.
• Writing good prompts — the same treatment for the prompt_template you just wrote, including the soul.md-vs-prompt split and the {{steps.x.y}} hyphen gotcha.
• Adding trust, one signal at a time — the full ladder this tutorial and the next one are climbing.
• Creating agents — the full agent.yaml reference, including scoping tools: to specific tool names and model fallbacks.