Retrieval pipeline

Hybrid retrieval fuses full-text and vector search into one ranked list. That primitive is excellent at recall and needs no score calibration — but it never reads the query against the text, it cannot follow the graph, and it packs context by chunk count rather than by what fits. The retrieval pipeline is a thin orchestrator that layers four research-backed stages over that primitive: query understanding, graph fusion, reranking, and a token-budget assembler. Each stage is a no-op when disabled or unavailable, so a workspace with no reranker key and understanding off gets exactly today's behavior — the upgrades are opt-in and degrade cleanly.

The five stages

1 · Query understanding

One structured judge call turns the raw question into a better retrieval query: a rewrite (the query actually run), up to three sub-queries for a bounded multi-query union that widens recall on multi-facet questions, advisory metadata filters (pathPrefix, entityTypes, docTitleContains), and an isAmbiguous flag. This deliberately replaces HyDE: research found that a direct rewrite plus decomposition beats embedding a speculative hypothetical document, at a fraction of the cost. Filters are advisory — a soft scope, never a hard exclusion that could zero out retrieval. The judge slot is the same model the answer policy already resolved; with no model, or the stage disabled, understanding returns nothing and the raw question is used.

2 · Hybrid retrieval + RRF (the primitive)

The effective query (rewrite, or the raw question) runs through retrieve — the two-channel hybrid described in Retrieval & grounded chat: websearch_to_tsquery full-text and pgvector cosine, combined with reciprocal-rank fusion (RRF, k = 60). The pipeline runs it wide — a candidate pool of ~40 chunks rather than the final answer budget — so the later reranking and assembly stages have real headroom to reorder and prune. Sub-queries retrieve into the same pool and merge by chunkId, keeping the max score.

3 · Graph fusion (GraphRAG local search)

RRF ranks text; it cannot follow a relationship. Graph fusion closes that gap with pure-SQL local search, reusing the same traversal service the graph channel uses:

1. collect the entities mentioned in the seed chunks' documents (canonical rollup, served-lifecycle only — draft, pending-review, and deleted entities are excluded), ranked by mention count, top 8;
2. expandFrontier over those seeds for a relevance-pruned 1–2 hop neighborhood (inverse-degree damping already stops hub nodes from dominating);
3. resolve the neighbor entities to their documents, then run a scoped retrieve over just those docs.

The result is chunks the raw lexical/vector pass missed because the connection was structural, not textual — an owner named in an adjacent runbook, a system two hops from the one you asked about. Those chunks merge into the pool tagged graphBoosted. It is deterministic and involves no LLM. Graph fusion respects a document scope when one is set: expansion still applies, bounded to the allowed documents.

4 · Reranking (cross-encoder)

RRF fuses rank positions; it never scores a chunk's text against the query. A cross-encoder reranker does exactly that — it reads (query, chunk) pairs jointly and emits a calibrated relevance score in [0, 1], which the pipeline uses to re-sort the pool. This is the single largest precision lever in modern RAG, and it is a pluggable adapter: Voyage rerank-2.5 (the default when a key is present), Cohere rerank-v3.5, a self-hosted bge-reranker-v2-m3 (ONNX), or none (identity passthrough). Provider details, auto-resolution, and cost are in Reranking. Any adapter error falls back to the fused RRF order — a reranker outage never fails the answer.

5 · Token-budget assembly

The old context step packed a fixed number of chunks. The assembler packs to a token budget (default 6,000, estimated at ~4 chars/token) with a hard cap of 20 chunks, which is what actually matters for a context window — a handful of long chunks and a dozen short ones cost very differently. Two more things happen here, both deterministic:

• MMR de-duplication.Greedy Maximal Marginal Relevance (λ = 0.7) trades relevance against redundancy using Jaccard similarity over normalized token shingles — no embeddings needed at assembly time, so it reproduces byte-for-byte. Near-duplicate chunks (Jaccard ≥ 0.9 to an already-selected chunk) are dropped outright, so three copies of the same paragraph don't eat the budget three times.
• Front-loading.The highest-scored chunk is always seated first, and stronger evidence leads — a direct mitigation for "lost in the middle," the finding that models attend best to the start and end of a long context and degrade in the middle.

The reranker score, when present, is the relevance signal MMR reads; otherwise it falls back to the fused score. The output is a RankedChunk[]capped at the caller's answer budget.

Degradation is the design

Every stage has an inactive path, and inactivity is defined by availability, not just a toggle: understanding without a judge model, a reranker that resolves to none (no key, self-host dep absent), and graph fusion disabled all count as inactive. When all three are inactive, the pipeline short-circuits to a plain retrieve(question, {limit, docIds})and skips the assembler entirely — the token-budget/MMR machinery only engages once an active stage has widened the pool. That is what makes "all stages off" identical to today'sretrieve + slice, and why eval reproducibility survives adding the pipeline.

The two non-deterministic stages — understanding and remote reranking — each have a deterministic-off path (no model; the none reranker), so an eval run can pin the pipeline to a fully reproducible configuration. Graph fusion and the assembler are deterministic always.

candidatePool: 40    // chunks fed to rerank/assemble after understanding + graph
graphAdded:    6     // distinct chunks graph fusion contributed to the pool
reranked:      true  // a real (non-none) reranker re-scored the pool
assembledTokens: 3820 // estimated tokens in the final assembled set

Those counts, plus the query rewrite and the reranker kind, ride along on a structured answer as an optional retrievalDebug block — see Structured query.

Research grounding

Like the answer policy, none of this is invented here — each stage maps to a published result: