Reranking

Hybrid retrieval fuses full-text and vector results by rank position(RRF) — it never scores a chunk's text against the query. A cross-encoder reranker is the second stage that does: it reads each (query, chunk) pair jointly and returns a calibrated relevance score, and the retrieval pipeline re-sorts the candidate pool by it. This is the single highest-leverage precision upgrade in the pipeline, and it is a pluggable adapter — Voyage, Cohere, a self-hosted open model, or a no-op — resolved per workspace from settings and environment.

Providers

Auto-resolution

The workspace reranker setting defaults to auto, resolved against the live environment at retrieval time (so adding a key takes effect without touching settings):

1. VOYAGE_API_KEY set → Voyage rerank-2.5 (override to rerank-2.5-lite with ELI_RERANK_MODEL).
2. else COHERE_API_KEY set → Cohere rerank-v3.5.
3. else → none (the fused RRF order, no network call).

A workspace can also pin a specific kind — voyage, cohere, onnx-bge, or none — in Settings → Retrieval. If a pinned provider is unavailable (its key is missing, or the self-host dependency is absent), resolution logs one warning and degrades to none rather than failing — retrieval always has a working reranker.

# Workspace default "auto": Voyage if VOYAGE_API_KEY, else Cohere, else none.
VOYAGE_API_KEY=pa-...             # default reranker (rerank-2.5), commercial
COHERE_API_KEY=...                # fallback reranker (rerank-v3.5), commercial
ELI_RERANK_MODEL=rerank-2.5-lite  # override the Voyage model (vs rerank-2.5)

The self-host path

For deployments where retrieval text must not leave the machine, set the workspace reranker to onnx-bge and install @huggingface/transformers. The adapter lazily loads onnx-community/bge-reranker-v2-m3-ONNX (Apache 2.0) and runs a text-classification pass locally in batches — no API key, no egress. The dependency is not bundled: until it is installed, onnx-bge is unavailable and resolution degrades to none with a logged warning, so selecting it early is harmless.

Where reranking is applied

• Grounded chat & structured query — the pipeline runs default-on, so answers are reranked whenever a real reranker resolves. Toggle it per send in the chat config panel, or per call via the workspace setting.
• Raw search — GET /api/v1/search stays on the raw primitive for back-compat and cost; pass ?rerank=true to opt one call into a reranked list. See Search & ask.
• MCP — kb_search takes an optional rerank argument (default false), so an agent can opt into reranked chunks when precision matters.

Cost & latency

Reranking adds one network round-trip over the candidate pool (~40 chunks, each truncated to ~2,000 characters before it is sent). The commercial APIs are inexpensive relative to a generation call — Voyage prices per token, Cohere per search unit — but they are not free, which is why raw /search defaults to off and only the answer pipelines rerank by default. Rough guidance:

• Voyage — the default; rerank-2.5-lite (via ELI_RERANK_MODEL) trades a little quality for lower latency and cost on high-volume workspaces.
• Cohere — comparable order of magnitude; a good fallback where a Cohere contract already exists.
• onnx-bge — no per-call fee at all; the cost is local compute (and cold-start model load on first use). Best when volume is high or egress is disallowed.

Settings

The reranker choice lives in the retrieval bag of workspace settings alongside the other pipeline knobs; inner keys override and clear independently, exactly like policy and chatDefaults:

{
  "retrieval": {
    "reranker": "auto",          // auto | voyage | cohere | onnx-bge | none
    "queryUnderstanding": true,  // 1 judge call: rewrite + sub-queries + filters
    "graphFusion": true,         // GraphRAG local-search expansion
    "tokenBudget": 6000,         // context assembler budget (chars/4 estimate)
    "candidatePool": 40          // width of the pool fed to rerank + assemble
  }
}