Use case: incident response

It is 2am and a service is paging. The on-call engineer does not need a wall of Slack scrollback — they need the runbook that resolves this class of incident, the accounts it is impacting, and a straight answer to how many incidents are open right now? — not a confident guess. This guide builds that assistant for Beacon Software, a B2B event-analytics platform, end to end across seven module slices: Intake to bring the postmortems and runbooks in, Atlas to type them into an incident graph, Warrant to certify the runbook, Conduit to wire in the live ops database, Lens to answer with citations, Lineage to trace any answer to its incident record, and Crucible + Ports to keep it accurate and ship it to on-call.

Prerequisites

• A workspace key with kb:write, kb:read, data:read, data:run, agents:run, and runs:read, exported as $ELI_KEY. See Getting started.
• Owner/admin on the workspace, to certify the runbook and verify the postmortem.
• The seeded beacon-ops Postgres connector reachable (the demo brings it up), so the live-data slices return rows.
• An enabled chat model, so /query can generate — otherwise 409 no-model-configured.

1 · Intake — bring the postmortems and runbooks in

On-call knowledge already exists — it is just scattered across postmortems, service catalogs, and runbooks. Get it into the workspace so it becomes searchable, citable documents. The seed writes the whole Beacon corpus for you, but the shape is the same for any note: post the markdown, commit its chunks and FTS index, then let embedding and entity extraction continue as detached work. Here is the INC-2043 postmortem going in:

curl -s -X POST https://your-host/api/v1/documents \
  -H "Authorization: Bearer $ELI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "path": "incidents/inc-2043-billing-outage.md",
    "title": "Incident — INC-2043 Billing Outage",
    "content": "# [[INC-2043]] — Billing Outage\n\nA Sev1 outage of the [[Billing Service]]. Invoicing and metering failed for ~90 minutes, impacting [[Vantage Retail]] and [[Brightline Logistics]]. Remediated via the [[Billing Failover Runbook]]; breached the [[Uptime SLA]]."
  }'

{ "docId": "01KY2043AAAAAAAAAAAAAAAAAA", "path": "incidents/inc-2043-billing-outage.md", "contentHash": "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824", "title": "Incident — INC-2043 Billing Outage" }

Notice the [[wikilinks]]: they remain source markup at 201. Detached enrichment can later let Atlas connect services, teams, incidents, runbooks, and accounts. The other seeded docs land the same way: runbooks/ingest-latency-runbook.md, services/ingest-gateway.md, accounts/acme-robotics.md, and the team roster in teams.md.

2 · Atlas — type the incident graph

Retrieval is only structured if the graph knows what things are. This workspace ships an ontology of six entity types — Team, Service, Account, Incident, Runbook, and System — wired together by four relations: owns, depends_on, impacted, and remediated_by. Pull the neighborhood around a key service — the Ingest Gateway — and the typed graph is right there:

curl -s "https://your-host/api/v1/entities/01KZINGESTGW00000000000000/neighborhood?depth=1" \
  -H "Authorization: Bearer $ELI_KEY"

{
  "entity": { "id": "01KZINGESTGW00000000000000", "name": "Ingest Gateway", "type": "Service", "authority": "machine_extracted" },
  "nodes": [
    { "id": "01KZPLATFORMTEAM000000000", "name": "Platform Team", "type": "Team" },
    { "id": "01KZAUTHSVC0000000000000A", "name": "Auth Service", "type": "Service" },
    { "id": "01KZMETRICSPIPE0000000000", "name": "Metrics Pipeline", "type": "Service" },
    { "id": "01KZINC2051000000000000AA", "name": "INC-2051", "type": "Incident" }
  ],
  "edges": [
    { "src": "Platform Team", "rel": "owns", "dst": "Ingest Gateway" },
    { "src": "Ingest Gateway", "rel": "depends_on", "dst": "Auth Service" },
    { "src": "Ingest Gateway", "rel": "depends_on", "dst": "Metrics Pipeline" },
    { "src": "INC-2051", "rel": "impacted", "dst": "Ingest Gateway" }
  ]
}

In one hop you can see that the Platform Team owns the gateway, that it depends on the Auth Service and Metrics Pipeline (so a degraded dependency is a latency suspect), and that INC-2051 impacted it. That structure is what makes the later answers precise instead of keyword soup.

3 · Warrant — certify the runbook, verify the postmortem

Extraction gives you concepts at authority machine_extracted — a starting point, not a source of truth. A runbook that on-call will follow at 2am has to be vouched for. In Warrant, propose the canonical remediation steps as a reviewable change, then certify the runbook — certified is the only tier the assistant states without hedging. Take the Ingest Latency Runbook:

curl -s -X POST https://your-host/api/v1/change-requests \
  -H "Authorization: Bearer $ELI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "kind": "update",
    "entityId": "01KZINGESTRUNBOOK00000000",
    "payload": { "description": "Ingest Latency Runbook (owned by Reliability Team): check Metrics Pipeline consumer lag, scale consumers, verify Auth Service health, then drain the backlog and page Platform On-Call." },
    "note": "Locking the on-call steps referenced during INC-2051"
  }'

{ "id": "01KZCR00000000000000000AA", "entityId": "01KZINGESTRUNBOOK00000000", "kind": "update", "status": "open" }

curl -s -X POST https://your-host/api/v1/entities/01KZINGESTRUNBOOK00000000/certify \
  -H "Authorization: Bearer $ELI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"note": "Validated by Reliability Team (Nadia Okafor) after INC-2051"}'

{ "authority": "certified", "lifecycleStatus": "published", "version": 3, "nextReviewAt": "2027-01-17T09:10:00.000Z" }

Do the same for the source record — verify the postmortem so its provenance carries a governance tier:

curl -s -X POST https://your-host/api/v1/documents/01KY2043AAAAAAAAAAAAAAAAAA/verify \
  -H "Authorization: Bearer $ELI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"verification":"certified","note":"Reviewed by incident commander (Nadia Okafor)"}'

{ "id": "01KY2043AAAAAAAAAAAAAAAAAA", "verification": "certified", "verifiedAt": "2026-07-22T09:12:00.000Z", "nextVerifyAt": "2026-12-29T00:00:00.000Z" }

4 · Conduit — wire in the live ops database

Documents answer "how do we fix this?" They cannot answer "how many incidents are open right now?" — that lives in the ops database, and it changes by the minute. The seed registers a read-only beacon-ops Postgres connector and a set of governed named queries — SELECT-only templates with typed params, so the model never writes raw SQL. List them:

curl -s https://your-host/api/v1/data/queries \
  -H "Authorization: Bearer $ELI_KEY"

{
  "items": [
    { "slug": "open-incidents-by-severity", "title": "Open incidents by severity", "connector": "beacon-ops", "params": [] },
    { "slug": "sla-breaches-for-service", "title": "SLA breaches for a service", "connector": "beacon-ops", "params": [{ "name": "service", "type": "string", "required": true }] },
    { "slug": "accounts-at-churn-risk", "title": "Accounts at churn risk", "connector": "beacon-ops", "params": [{ "name": "threshold", "type": "number", "required": true }] }
  ],
  "total": 3
}

Run the no-parameter one to get live open-incident counts. The executor enforces read-only SQL, session hardening, and row caps, and records a data_calls provenance row — the [Dn] anchor Lens will cite:

curl -s -X POST https://your-host/api/v1/data/queries/open-incidents-by-severity/run \
  -H "Authorization: Bearer $ELI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"params":{}}'

{
  "dataCallId": "01KZDATACALL000000000000A",
  "connectorId": "01KZBEACONOPS00000000000A",
  "queryId": "01KZQUERYOPENINC000000000",
  "columns": ["severity", "open_count"],
  "rows": [["sev1", 1], ["sev2", 2]],
  "rowCount": 2,
  "durationMs": 18,
  "truncated": false
}

5 · Lens — the on-call engineer asks, and gets a cited answer

Now the payoff. An engineer asks in plain language; Lens retrieves documents and runs the bound live query, grounds the generation, and returns an answer with inline [Sn] document and [Dn] data citations plus an answer-policy verdict. Set includeData: true so the ops database is in scope. Ask the live question verbatim:

curl -s -X POST https://your-host/api/v1/query \
  -H "Authorization: Bearer $ELI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"question":"How many incidents are open right now, by severity?","includeData":true}'

{
  "answer": "There are three open incidents right now — one Sev1 and two Sev2 [D1]. Live counts come from the ops database via the Observability Stack, which is the source of truth for every incident record [S1].",
  "claims": [
    { "text": "There are three open incidents: one Sev1 and two Sev2", "citations": ["D1"] },
    { "text": "The Observability Stack is the source of truth for incident records", "citations": ["S1"] }
  ],
  "sources": [
    { "id": "S1", "docId": "01KZOBSSTACK0000000000000A", "docTitle": "System — Observability Stack", "docPath": "systems/observability-stack.md", "chunkId": "01KZOBSCHUNK00000000000AA" }
  ],
  "data": [
    { "id": "D1", "connector": "beacon-ops", "query": "open-incidents-by-severity", "dataCallId": "01KZDATACALL000000000000A", "rowCount": 2, "executedAt": "2026-07-22T09:20:11.004Z" }
  ],
  "entities": [ { "id": "01KZOBSSTACK0000000000000A", "name": "Observability Stack", "type": "System", "authority": "certified" } ],
  "policy": { "verdict": "answer", "confidence": 0.81, "reasons": ["confidence_above_answer_floor"] },
  "groundedness": { "verified": 2, "total": 2 },
  "model": { "chat": "claude-sonnet-4-5", "judge": "claude-haiku-4-5" }
}

The policy block is the trust signal: answer at 0.81 because a live data call cleared the answer floor. Ask something no record backs — "Will the ingest pipeline stay healthy next week?" — and the policy engine returns abstain_with_pointers at confidence 0.1 (reason no_relevant_chunks) instead of forecasting a number it cannot support. A borderline one — "Which accounts are at churn risk?" — lands on answer_with_caveat (confidence_in_caveat_band). That is exactly the behavior on-call demands: cite it, caveat it, or abstain. The same pipeline is a native MCPtool, so it drops into an engineer's Claude or a Slack incident channel.

6 · Lineage — trace it to the incident record

An incident commander reviewing the assistant asks the fair question: where is this coming from, and can I trust it? The answer cited a document source and a live data call. Pull the postmortem's lineage and read the whole trail in one call — origin, governance, and the downstream answers it is grounding right now:

curl -s https://your-host/api/v1/documents/01KY2043AAAAAAAAAAAAAAAAAA/lineage \
  -H "Authorization: Bearer $ELI_KEY" | jq '{origin: .origin.kind, verification: .governance.verification, citedBy: .downstream.citedByAnswers, impactedEntities: .downstream.impactedEntities}'

{
  "origin": "vault",
  "verification": "certified",
  "citedBy": 9,
  "impactedEntities": ["Billing Service", "Vantage Retail", "Brightline Logistics", "Billing Failover Runbook", "Uptime SLA"]
}

Closed loop: the answer traces to the certified INC-2043 postmortem, the document is certified, it is currently grounding 9 answers, and its derived concepts name the exact blast radius — the Billing Service, the impacted accounts Vantage Retail and Brightline Logistics, and the Billing Failover Runbook that resolved it. The commander did not have to trust the model; they read the pedigree. Edit the postmortem and you know precisely which answers and cache entries invalidate.

7 · Crucible + Ports — lock the accuracy in, then ship it

Runbooks and ownership change constantly, and a wrong answer during an incident is expensive. Capture the questions on-call actually asks as a golden set so Crucible can prove the assistant still answers them after every content change. The seed ships three — "Which team owns the Billing Service?" (Payments Team, Wei Chen), "Which runbook resolves ingest latency incidents?" (Ingest Latency Runbook), and "Why did INC-2043 breach the Uptime SLA?". Anchor the runbook question to the chunk that should ground it:

curl -s -X POST https://your-host/api/v1/qrels \
  -H "Authorization: Bearer $ELI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "queryId": "01KZGOLDENRUNBOOK00000000",
    "chunkId": "01KZRUNBOOKCHUNK000000000",
    "docId": "01KZRUNBOOKDOC00000000000",
    "grade": 3,
    "source": "provenance"
  }'

{ "id": "01KZQREL0000000000000000A", "queryId": "01KZGOLDENRUNBOOK00000000", "grade": 3, "source": "provenance" }

The seed also registers the SRE support Q&A baseline eval config and runs it once. Read the metrics for that run — retrieval and abstention numbers are deterministic; RAGAS numbers carry the judge model that produced them:

curl -s https://your-host/api/v1/evals/runs/01KZRUNA00000000000000000/metrics \
  -H "Authorization: Bearer $ELI_KEY"

{
  "runId": "01KZRUNA00000000000000000",
  "n": 3,
  "retrieval": { "recallAtK": 0.74, "ndcgAt10": 0.71, "mrr": 0.79, "hitAtK": 0.93 },
  "ragas": { "faithfulness": 0.91, "responseRelevancy": 0.84, "contextPrecision": 0.66 },
  "abstention": { "truthfulness": 0.62, "abstainRate": 0.11, "aurc": 0.074 }
}

When a metric drops after a runbook edit, the golden set names exactly which question regressed — and promoting that miss back into the set turns it into a permanent test. That is the dashed loop in the diagram: Crucible's regressions steer the next round of Warrant curation. Compare two configs with a two-gate significance test in Measure & compare.

The leverage

Seven slices, one knowledge base, one audit trail. The on-call engineer gets a fast, cited answer — even a live count straight from the ops database; the incident commander can trace any answer to a certified postmortem and see its blast radius; and the whole thing is measured, so accuracy is a number you watch rather than a hope. Swap incident response for security response, compliance, or customer reliability and the shape is identical — the platform doesn't care what the records are about, only that every answer is grounded, governed, traceable, and measured.