Tune retrieval quality

The retrieval pipeline ships default-on for grounded chat and the structured query API, and it degrades to raw hybrid retrieval wherever a stage is unavailable. This guide walks the levers that matter: turning on a reranker, opting raw search into reranked results, deciding when graph fusion pays for itself, sizing the token budget, and reading the retrieval trace to confirm a stage actually fired.

Prerequisites

• Owner/admin on the workspace — pipeline settings live in Settings → Retrieval, owner/admin-gated like all workspace settings.
• A key with kb:read (for /search) or agents:run (for /query), exported as $ELI_KEY. See Getting started.
• For a commercial reranker, a VOYAGE_API_KEY or COHERE_API_KEY in the deployment environment. No key is required — the pipeline runs fine without one.

1 · Turn on a reranker

Reranking is the biggest single precision win. Set one provider key in the environment and the workspace default (auto) picks it up at retrieval time — Voyage if VOYAGE_API_KEY is set, else Cohere, else the no-op passthrough:

VOYAGE_API_KEY=pa-...   # auto → Voyage rerank-2.5 (the default reranker)

No settings change is needed for auto to start reranking. To pin a specific provider (or turn it off), set the reranker key in Settings → Retrieval. Inner keys of the retrieval bag override and clear independently:

{
  "retrieval": { "reranker": "voyage" }
}

2 · Opt raw search into reranked results

GET /api/v1/search stays on the raw RRF primitive by default — it is the cheap, model-free evidence endpoint, and reranking every autocomplete keystroke would be wasteful. When you want a reranked list for one call, pass ?rerank=true:

curl -s "https://your-host/api/v1/search?q=stuck%20orders&limit=5&rerank=true" \
  -H "Authorization: Bearer $ELI_KEY"

If the workspace reranker resolves to none (no key), ?rerank=true is a harmless no-op and you get the fused order back. Agents over MCP get the same opt-in via the rerank argument on kb_search (default false).

3 · Decide when graph fusion earns its keep

Graph fusion runs GraphRAG local search — it seeds from the entities mentioned in the top hits, expands 1–2 hops over the entity graph, and pulls chunks from neighbor documents the raw text pass missed. It is on by default and adds no LLM cost, but it only helps when your corpus has a graph to follow. Leave it on when:

• extraction has built a real entity graph (systems, people, concepts with relations) — the manifest's hub-concept counts are a quick proxy;
• questions are relational— "who owns the service that depends on X," "what else did this incident touch" — where the answer lives one hop from the obvious document.

Turn it off for a thin or nearly graph-free corpus (a flat folder of notes with little extraction), where expansion adds candidates without adding signal:

{
  "retrieval": { "graphFusion": false }
}

Graph fusion respects a query's document scope: when a chat send or query restricts to certain documents, expansion stays inside that scope. Only served entities seed and expand — draft and pending-review entities never leak into retrieval.

4 · Size the token budget

The assembler packs chunks until a token budget is hit (default 6,000, ~4 chars/token), de-duplicating with MMR and front-loading the strongest evidence. Raise it to give the model more evidence on a large-context chat model; lower it to cut cost and sharpen focus on a small one. The hard cap is 20 chunks regardless:

{
  "retrieval": { "tokenBudget": 9000 }
}

5 · Query understanding

Query understanding spends one judge call to rewrite the question, decompose it into up to three sub-queries, and derive advisory metadata filters. It is on by default and uses the same judge model the answer policy already resolved, so it adds no new model configuration — but it does add one call per question. Turn it off to make retrieval fully deterministic on the query side (useful for cost-sensitive or reproducibility-sensitive workspaces); the rewrite is simply skipped and the raw question is used:

{
  "retrieval": { "queryUnderstanding": false }
}

6 · Read the retrieval trace

Confirm a stage actually fired. The structured query API surfaces an optional retrievalDebugblock — the query rewrite, the per-stage counts, and the reranker kind — so you can see the pipeline's work instead of guessing:

curl -s -X POST https://your-host/api/v1/query \
  -H "Authorization: Bearer $ELI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"question":"Who owns the service behind the March order incident?"}'

{
  "answer": "…",
  "retrievalDebug": {
    "understanding": {
      "rewrite": "owner of the service implicated in the March 2026 order incident",
      "subQueries": ["March 2026 order incident service", "service ownership order platform"],
      "isAmbiguous": false
    },
    "stages": { "candidatePool": 40, "graphAdded": 6, "reranked": true, "assembledTokens": 3820 },
    "reranker": "voyage"
  }
}

Read it as a checklist: a non-empty rewrite means understanding ran; graphAdded > 0 means graph fusion contributed chunks; reranked: true with a provider name means a real reranker re-scored the pool (reranker: "none" means it degraded — check your key). If every stage reads inactive, the pipeline returned raw hybrid retrieval, which is the correct, byte-identical baseline.