Overview

System diagrams

eli.ai is eight systems that compose. This page shows how they fit together — first the shape of the machine as one plate, then the master flow of a document and a question through the whole platform, then a detailed internal architecture diagram for each system, with every node mapped to a real module or table. For the narrative overview, see the capability map; to see the systems in action, seed a demo workspace.

The shape of the machine

Every step of the grounded answer path, listed in order below, followed by all eight systems and every documented handoff between them. On large screens the plate also draws those systems and handoffs as a map, with this path picked out.

Intake

Atlas

Warrant

Lineage

Conduit

Lens

Crucible

Ports

A cited answer that respects source permissions and stops when certified evidence ends.

Intake
Atlas
Warrant
Lineage
Conduit
Lens
Crucible
Ports

All system handoffs

Intake → Atlas Normalized documents feed typed entity and relation extraction with sentence evidence.
Atlas → Warrant Graph concepts receive authority tiers, lifecycle state, ownership, and governed change.
Atlas → Lens Graph neighborhoods join lexical and vector retrieval before reranking.
Warrant → Lens Content ACLs and certification state constrain what the answer pipeline may state.
Conduit → Lens Entitled named queries contribute live rows and auditable Data Citations.
Lens → Ports Grounded results reach the workspace, REST clients, MCP hosts, and durable agents.
Lens → Crucible The real answer pipeline runs against golden sets and measured quality gates.
Crucible → Warrant Significance-gated evaluation results govern consequential configuration changes.
Lens → Lineage Answer claims, source revisions, data calls, and policy verdicts remain traceable.
Lineage → Intake Reverse provenance resolves a result back to the exact ingested source revision.

The plate is the same eight systems and the same handoffs as the charts below, said without any table names: every documented edge is drawn, the grounded-answer path is picked out, and the “All system handoffs” list gives the mechanism behind each edge in words. Read it first if the flowcharts below are more machinery than you need — they are the implementation view of this picture, not a different one.

How the eight systems interconnect

A document enters through Intake, becomes the Atlas graph, and is governed by Warrant. A question is answered by Lens, which retrieves over that graph and the chunks, joins live systems of record through Conduit, and gates the answer on the policy engine — then hands it to a consumer through Ports. Crucible measures the answers and steers the next round of curation, and Lineage makes the whole thing traceable end to end. The dashed edges are the feedback loops that keep it honest.

End-to-end interconnect

The document pipeline (Intake → Atlas → Warrant), the question pipeline (Lens over the graph + Conduit live data, gated by the answer policy, out through Ports), and the two feedback loops — Crucible regressions steering Warrant curation, and every answer traceable back to its source through Lineage.

Downloads

Concepts

System diagrams
End-to-end interconnect
INTAKE — connect & ingest
ATLAS — the semantic layer
WARRANT — govern & certify
CONDUIT — the live-data layer
LENS — retrieve & answer
CRUCIBLE — evaluate & measure
PORTS — programmatic surfaces
Master

Keywords

chat · agent · · MCP
Postgres · MySQL · Snowflake
REST · MCP · durable agents
data_calls provenance
sync / saveDocument
documents · chunks
knowledge graph
structured records
answer policy
entity bindings
systems of record
hybrid retrieval
grounded generation
groundedness judge
golden sets · eval runs
drift · coverage
webhooks · schedules
spans every system
certify / govern
graph + [Sn] chunks
[Dn] rows
pre / post gate
trace answer → source
chunk → FTS tsvector + embeddings
wikilinks + LLM extraction
entities · relations · ontology
matrices · deals · rules · factors
projected into the corpus
authority tiers · certify
verify · change requests
pre / post gate · calibration
→ named read-only queries
per-user entitlements (fail closed)
FTS + vector · RRF · rerank
metrics · acceptance gates
LINEAGE — provenance & audit
append-only revisions · blast radius
corpus projection · [Sn]-citable
regressions steer curation
chunk_embeddings

Source and generation provenance

Status: current

Generated at: 2026-08-12T23:38:15.207Z

Source hash: 6c649318916666d8620ba4482045b32e1a5092503cedd184cc2ed7e335055147

Metadata payload hash: 73fe22adc41c5e5272597e54419e2d329b49eaf8459a0cbef064ba62fc2df52f

Canonical appearance

src/app/(docs)/docs/concepts/system-diagrams/page.tsx:59 route /docs/concepts/system-diagrams

All appearances

mirror — docs/architecture/system-diagrams.md:28
canonical — src/app/(docs)/docs/concepts/system-diagrams/page.tsx:59 route /docs/concepts/system-diagrams

Generation versions

App: eli-ai 0.1.0

Mermaid: 11.16.0 · Mermaid CLI: 11.16.0

Node: v26.3.1 · Yarn: 4.17.1

Renderer config hash: 68c10966fe84406ee626034d58bfabd555df9f65f691204b7c46db24038da101

Renderer theme hash: c80287a78d80ad63d27bd5ca348b2ef9a7e2f44da289e436be6484ea28a1b033

Adapter versions: diagramGenerator=2, drawioFlowchart=1, drawioGantt=1, drawioSequence=1, drawioState=1

Full sidecar JSON: end-to-end-interconnect-6c649318.json

Each system, inside

Eight diagrams, one per system — inputs on the way in, the internal stages and stores, and what each hands to the next. Every node names a real module or table.

1IntakeConnect & ingest

Confluence, SharePoint, Slack, and markdown-upload sources share one contract. Remote connector requests use the shared address-pinned, redirect-revalidating, bounded HTTP client. A database advisory lock serializes every sync kind for a workspace/source across queues and workers. The crawl awaits each idempotent ingestConnectorDocument upsert before persisting the page checkpoint, so acknowledged progress cannot outrun document/revision state. Unchanged content skips re-chunk/re-embed; a real change appends document_revisions, writes ACL principals, reconciles chunks/FTS, and dispatches embedding/extraction. Manual text import uses saveDocument/importMany and converges on the same derived stores through its caller-owned enrichment path.

Intake — connect and ingest

Intake serializes each source, ingests every normalized document before advancing its checkpoint, writes revision/ACL metadata, and dispatches searchable chunk enrichment.

Downloads

Concepts

System diagrams
Ingest + index
Intake — connect and ingest
Sync + normalize · worker
Inputs
Intake

Keywords

REST API · UI · schedule
connector_source types
upsert by (source_id, external_id)
documents + document_revisions
Confluence · SharePoint
Slack · Markdown upload
normalize →
contentHash /
sourceVersion changed?
embedChunks → embedMany
batch of 64
enqueue SyncJob
start / finish ledger
await idempotent ingest
unchanged, live
chunks + tsvector
pgvector rows
enqueueExtract · chunks
tsvector FTS
vector kNN
Manual edit · bulk import · binary upload
saveDocument · importMany · normalize
per-source DB advisory lock
all sync kinds / all workers
syncWithConnector · drainCrawl
fullLoad / poll → DocBatch
skip re-chunk / re-embed
chunkMarkdown → persistChunks
Atlas · entity extraction
Lens · hybrid FTS + vector
confirmed page → checkpoint
sync_state
acl_principals
chunk_embeddings
embedding_spaces
/api/v1/connectors
id/sync
NormalizedDocument
ingestConnectorDocument
GIN

Source and generation provenance

Status: current

Generated at: 2026-08-12T23:39:30.559Z

Source hash: 98d98cb8b75042b0195fab5ad998bd0804f6e9ad5add341e71b3eaa7ca95b21e

Metadata payload hash: c9615427ae6b8b04b9d327b83168a6faaedb5c5d38111ffbb8ed8062d73070e7

Canonical appearance

src/app/(docs)/docs/concepts/system-diagrams/page.tsx:111 route /docs/concepts/system-diagrams

All appearances

mirror — docs/architecture/system-diagrams.md:97
canonical — src/app/(docs)/docs/concepts/system-diagrams/page.tsx:111 route /docs/concepts/system-diagrams

Generation versions

App: eli-ai 0.1.0

Mermaid: 11.16.0 · Mermaid CLI: 11.16.0

Node: v26.3.1 · Yarn: 4.17.1

Renderer config hash: 68c10966fe84406ee626034d58bfabd555df9f65f691204b7c46db24038da101

Renderer theme hash: c80287a78d80ad63d27bd5ca348b2ef9a7e2f44da289e436be6484ea28a1b033

Adapter versions: diagramGenerator=2, drawioFlowchart=1, drawioGantt=1, drawioSequence=1, drawioState=1

Full sidecar JSON: intake-connect-and-ingest-98d98cb8.json

Key modules & tables

src/server/connectors/sync.ts — advisory lock, inline drainCrawl ingestion before checkpoint, revisions, prune and permission refresh
src/server/connectors/sources/http.ts + src/server/network — validated, address-pinned connector HTTP with redirect, time and size bounds
src/server/connectors/registry.ts + types.ts — Connector fullLoad / poll / listIds / getPermissions contract
src/server/vault/index.ts + src/server/ingest/bulk.ts — recoverable manual save and text bulk-import path
src/server/ingest/chunking.ts — chunkMarkdown + persistChunks; generated tsvector/GIN index
src/server/ai/embeddings.ts — embedChunks → embedMany into the active embedding space
Tables: connector_sources, sync_runs, documents, document_revisions, chunks, chunk_embeddings, embedding_spaces

Module reference: Intake →Deep dive: Intake →

2AtlasThe semantic layer

Chunks enter Atlas through zero-model [[wikilink]] resolution and capped co-occurrence edges. Co-occurrence carries explicit synthetic provenance and replaces its own document pair set idempotently. New model work uses the stately extract-v2 queue: the worker snapshots the exact document, ordered chunks, and ontology, completes every provider call before graph mutation, then locks and revalidates those inputs plus latest-run ownership. One transaction wipes the previous model projection, applies every chunk, rebuilds canonical edges, and completes the run, so the last-good graph remains visible on failure or stale work. Missing/superseded runs are acknowledged; content or ontology changes retry from current state. Document deletion purges graph provenance before chunks in the tombstone transaction, retiring only machine orphans while preserving manual/asserted structure. Readers use entities plus canonical_relations, the maintained serve-time edge projection.

Atlas — semantic graph

Evidenced deterministic links, atomic ontology-typed extraction, and human curation converge on canonical_relations.

Downloads

Concepts

System diagrams
Atlas — semantic graph
Atlas
Graph construction — 3 layers + human curation

Keywords

entity_types · relation_types
kb_entity_lookup · kb_graph_query
chunks · doc title
from Intake ingest
ontology + constraints
final transaction
stale outcome
serve-time edge table
on save
then, same save
locked exact snapshot
snapshot/owner changed
one-TX replace
→ Lens · Ports
zero-LLM wikilink → entities + mentions
explicit evidence · idempotent per doc
snapshot doc + ordered chunks + ontology
all provider calls before graph mutation
exact revalidate · latest-run owner
wipe + apply all chunks + finish run
human entities · merges · revisions
extract- · stately worker
exact snapshot + constraints
latest owner + rejection signatures
entity_aliases
relation_evidence
extraction_runs
rejection_tombstones
entity_revisions
merge_log
canonical_relations
domain/range
missing/superseded
content/ontology
schema/prompt
resolveWikilinks()
extractDocument()
extractChunkFacts()
acknowledge
retry

Source and generation provenance

Status: current

Generated at: 2026-08-12T23:36:52.603Z

Source hash: 987690f873b648a9de4876fbbb4d69d92eedb40f21ce589d804499a274372385

Metadata payload hash: e3fac60f80287daca7f79cbef121c027a9b6ecab2df4635021ba5b8ae251a11c

Canonical appearance

src/app/(docs)/docs/concepts/system-diagrams/page.tsx:162 route /docs/concepts/system-diagrams

All appearances

mirror — docs/architecture/system-diagrams.md:180
canonical — src/app/(docs)/docs/concepts/system-diagrams/page.tsx:162 route /docs/concepts/system-diagrams

Generation versions

App: eli-ai 0.1.0

Mermaid: 11.16.0 · Mermaid CLI: 11.16.0

Node: v26.3.1 · Yarn: 4.17.1

Renderer config hash: 68c10966fe84406ee626034d58bfabd555df9f65f691204b7c46db24038da101

Renderer theme hash: c80287a78d80ad63d27bd5ca348b2ef9a7e2f44da289e436be6484ea28a1b033

Adapter versions: diagramGenerator=2, drawioFlowchart=1, drawioGantt=1, drawioSequence=1, drawioState=1

Full sidecar JSON: atlas-semantic-graph-987690f8.json

Key modules & tables

src/server/graph/extraction.ts — wikilinks, provider-before-mutation extraction, exact snapshot/owner gate, one-TX replace, delete purge and fact tombstones
src/server/graph/cooccurrence.ts — capped deterministic related_to edges with explicit idempotent provenance
src/server/jobs/index.ts + src/worker/index.ts — stately extract-v2 producer/worker contract and legacy queue drain
src/server/graph/ontology.ts + ontology-admin.ts — vocabulary, constraints, templates and re-extraction
src/server/graph/manual.ts + curation.ts + review.ts — human graph edits, revisions and duplicate merges
src/server/graph/tools.ts + query.ts — entity/graph reads over canonical entities and edges
Tables: entities, aliases, mentions, relations, evidence, canonical_relations, extraction_runs, revisions

Module reference: Atlas →Deep dive: Atlas →

3WarrantGovern & certify

Two write paths feed the graph: schema-level and destructive edits are gated through change-requests.ts (propose → owner/steward approve → apply), while every mutation and document verification is stamped into append-only entity_revisions (PROV-O) and documents.verification, escalating each concept up the authority ladder machine_extracted → asserted → curated → certified. At answer time that same governance state flows into the policy engine: router.preGate screens retrieval before generation (score floor, deterministic entity ambiguity, an optional LLM relevance/ambiguity call), and router.postGate scores calibrated confidence from confidence.ts — hard-capping it below the answer floor when only unverified sources are cited and adding caveats from evidence_conflicts. The pre- and post-verdicts (proceed/clarify/abstain and answer/answer_with_caveat/abstain) are logged append-only to policy_decisions, which closes the tuning loop: calibration.ts labels those decisions from reviewer feedback, refits a logistic model, and activates one policy_calibrations row that resolveActiveCoeffs feeds back into every future postGate.

Warrant — governance and answer policy

Warrant governs two things at once: the authority and revision history of every concept and document, and the pre/post-gate answer-policy engine whose calibrated confidence — hard-capped for unverified sources — decides answer, caveat, abstain, or clarify.

Downloads

Concepts

System diagrams
Warrant — governance and answer policy
Concept + document authority — curation
Warrant
Answer-policy engine — pre / post gate + calibration

Keywords

machine_extracted → asserted
calibrated confidence (confidence.ts)
clarify / abstain_with_pointers
answer / answer_with_caveat / abstain
citation_enforcement decision
conflicting_evidence caveat
authority tiers
→ curated → certified
PROV-O append-only
approved → apply
one active
→ Lens · Ports
citation enforcement
off / shadow / enforce
fit → draft
extraction · authors · stewards / owners
proposeChange → approve (Track A gate)
certify · markReviewed · setAuthority
verifyDocument (Glean tiers)
question + fused chunks (Lens)
score floor · entity ambiguity · LLM relevance
recordFeedback → recalibrate → activate
governed answer + PolicyBlock
proceed → generate answer
retry once → deterministic abstain
reviewer labels feedback
only-unverified → confidence cap
certified → higher confidence
concept_change_requests
entity_revisions
documents.verification
router.preGate
router.postGate
policy_decisions
policy_calibrations
evidence_conflicts
recordRevision
append-only
resolveActiveCoeffs

Source and generation provenance

Status: current

Generated at: 2026-08-12T23:43:28.542Z

Source hash: 2f146ef27a582b7d2c33112796d692006b9a9d37498d648b00c81cb5bb1aa8b2

Metadata payload hash: 7c5955b9a70b0f42111477ead425be216a12e9ec1b6370a147ba055bb8598432

Canonical appearance

src/app/(docs)/docs/concepts/system-diagrams/page.tsx:212 route /docs/concepts/system-diagrams

All appearances

mirror — docs/architecture/system-diagrams.md:272
canonical — src/app/(docs)/docs/concepts/system-diagrams/page.tsx:212 route /docs/concepts/system-diagrams

Generation versions

App: eli-ai 0.1.0

Mermaid: 11.16.0 · Mermaid CLI: 11.16.0

Node: v26.3.1 · Yarn: 4.17.1

Renderer config hash: 68c10966fe84406ee626034d58bfabd555df9f65f691204b7c46db24038da101

Renderer theme hash: c80287a78d80ad63d27bd5ca348b2ef9a7e2f44da289e436be6484ea28a1b033

Adapter versions: diagramGenerator=2, drawioFlowchart=1, drawioGantt=1, drawioSequence=1, drawioState=1

Full sidecar JSON: warrant-governance-and-answer-policy-2f146ef2.json

Key modules & tables

src/server/graph/curation.ts — recordRevision, certifyEntity, markReviewed, setAuthority; authority tiers (machine_extracted→asserted→curated→certified) + lifecycle + review SLAs
src/server/graph/change-requests.ts — proposeChange / approveChangeRequest / rejectChangeRequest (Track A gated review, claim → apply → stamp appliedAt)
src/server/vault/verification.ts — verifyDocument (Glean verified/certified tiers, next_verify_at)
src/server/policy/router.ts — preGate (floor · entity ambiguity · LLM relevance) and postGate (4-verdict engine + Glean confidence cap)
src/server/policy/confidence.ts + calibration.ts — logistic calibrated confidence + feedback→recalibrate→activate tuning loop (resolveActiveCoeffs)
Tables: entity_revisions, concept_change_requests, documents.verification, policy_decisions, policy_calibrations, evidence_conflicts (all workspace-scoped, FORCE-RLS)

Module reference: Warrant →Deep dive: Warrant →

4LineageProvenance & traceability

documentLineage(workspaceId, docId) performs ten staged, tenant-isolated SQL reads (the recent-sync read is conditional) for one live document. It returns origin, verification/review state, the total and newest 50 document revision summaries, the newest 50 document audit events, derived chunk/entity/relation counts, cited-answer count, dependent cache entries, and impacted canonical entities. Entity revision history remains available through the separate entity revisions API and is not embedded in DocumentLineage. REST, MCP, the workspace BFF, and the document UI consume the same bounded payload.

Lineage — provenance and blast radius

DocumentLineage unifies origin, governance, bounded revision/audit summaries, derivation, and downstream blast radius without a model call.

Downloads

Concepts

System diagrams
Lineage — provenance and blast radius
Append-only history · PROV-O
Downstream · Lens — blast radius
Lineage
Origin · Intake — where it came from
Derived · Atlas — knowledge it produced

Keywords

separate entity-history API
FORCE-RLS · deterministic · no LLM
origin: source_id
type · external_id · source_version
path · version · content_hash
verification tier · next_verify_at
relation_evidence → relations
recent runs · errors
Governance · Warrant
DB trigger
mentions → entities
governance cols
revisions {total,items}
linked from entities
not embedded
content+meta snapshots · activity
canonical concepts · authority · lifecycle
persisted answers citing this doc
documentLineage(ws, docId)
origin · governance · revisions {total,items}
derived · downstream · audit
entities · impactedEntities
connector_sources
sync_runs
document_revisions
entity_revisions
semantic_cache_entries
documents
reviewOverdue
chunks
messages
dependencies
DocumentLineage
total,items
recentSyncRuns
document
audit[]
chunkCount
relationCount
citedByAnswers

Source and generation provenance

Status: current

Generated at: 2026-08-12T23:39:57.028Z

Source hash: caf14625e044e75d702a8c7f42b68ac9231fc199599a1e5669b0b9b9a6f115fc

Metadata payload hash: 13360a04c3862414d72e2dba73d9d05871a566e9ac3ce097aef722ee8f598522

Canonical appearance

src/app/(docs)/docs/concepts/system-diagrams/page.tsx:256 route /docs/concepts/system-diagrams

All appearances

mirror — docs/architecture/system-diagrams.md:341
canonical — src/app/(docs)/docs/concepts/system-diagrams/page.tsx:256 route /docs/concepts/system-diagrams

Generation versions

App: eli-ai 0.1.0

Mermaid: 11.16.0 · Mermaid CLI: 11.16.0

Node: v26.3.1 · Yarn: 4.17.1

Renderer config hash: 68c10966fe84406ee626034d58bfabd555df9f65f691204b7c46db24038da101

Renderer theme hash: c80287a78d80ad63d27bd5ca348b2ef9a7e2f44da289e436be6484ea28a1b033

Adapter versions: diagramGenerator=2, drawioFlowchart=1, drawioGantt=1, drawioSequence=1, drawioState=1

Full sidecar JSON: lineage-provenance-and-blast-radius-caf14625.json

Key modules & tables

src/server/lineage/index.ts — deterministic 10-stage assembler; no LLM, network, or filesystem I/O
src/server/vault/revisions.ts — append-only document revisions with snapshot/diff/activity
audit_log — bounded document.* event trail
src/server/graph/curation.ts + entity revision endpoint — separate concept history
documents + connector_sources + sync_runs — origin and governance
messages, semantic-cache dependencies, mentions/entities — downstream impact

Module reference: Lineage →Deep dive: Lineage →

5ConduitThe live-data layer

Conduit is the bridge from the semantic graph to systems of record. When a question touches an entity, resolveBindingsForEntity (bindings.ts) gathers the entity's own bindings plus the ones inherited from its type, buildParams materializes each parameter from the binding's paramMap (entity_name, const, or model-supplied), and executeNamedQuery (executor.ts) loads the matching data_queries domain component and its data_connectors row inside one short RLS transaction, then coerces params against each DataParamSpec. The governed tail runs assertReadOnlySql (single SELECT/WITH only), decrypts the credential at call time, and dispatches to the engine adapter (postgres/mysql/snowflake) which rewrites $1..$n placeholders to the driver bind style and caps the result by maxRows and timeoutMs — all strictly outside any tenant transaction, since connector SQL is network I/O. Every execution appends an append-only data_calls provenance row (executed SQL, params, row count, duration, status), and that row's dataCallId is the anchor behind the [Dn] data citation handed into the grounded answer.

Conduit — governed live data

Conduit: a bound entity plus a question resolves through entity_bindings to a governed named query, runs read-only against a system of record via the per-engine adapter, and every call — ok, error, timeout, or blocked — appends a data_calls row that becomes a [Dn] citation.

Downloads

Concepts

System diagrams
Conduit — governed live data
CONDUIT — bind, govern, execute
Inputs
Conduit

Keywords

named SQL template · $1..$n · params
chat · agent · · MCP
Postgres · MySQL · Snowflake
entity_name · const · model
kb_data: touches bound entity
bound entity / type
from Atlas graph
single SELECT / WITH?
→ query + paramMap
append-only provenance
reads bindings
materialized params
caps + creds
denied · fail closed
ok / not scoped
normalized JSON rows
ok · error · timeout
entity + inherited type bindings
load query+connector · coerceParams
decryptSecret · getAdapter
rewriteDollarParams · wrapWithRowLimit
encrypted creds · maxRows · timeoutMs
sql · params · rows · status
result snapshot + hash · envelope
external system of record
sqlTemplate + param specs
principal grants cover value
user × param × value grants
capped SELECT · $n→? binds
entity_bindings
data_queries
data_connectors
data_calls
entitlement_param?
data_entitlements
/api/v1/query
entity/type
question
buildParams
executeNamedQuery

Source and generation provenance

Status: current

Generated at: 2026-08-12T23:37:30.526Z

Source hash: cda4d617212fac72247c1c39543f5be26d5be55ec58744d622db822200a6f132

Metadata payload hash: fe20151aec2109ae985acd4f8116088584a6138c6638d001a90361ec86b73a2f

Canonical appearance

src/app/(docs)/docs/concepts/system-diagrams/page.tsx:295 route /docs/concepts/system-diagrams

All appearances

mirror — docs/architecture/system-diagrams.md:406
canonical — src/app/(docs)/docs/concepts/system-diagrams/page.tsx:295 route /docs/concepts/system-diagrams

Generation versions

App: eli-ai 0.1.0

Mermaid: 11.16.0 · Mermaid CLI: 11.16.0

Node: v26.3.1 · Yarn: 4.17.1

Renderer config hash: 68c10966fe84406ee626034d58bfabd555df9f65f691204b7c46db24038da101

Renderer theme hash: c80287a78d80ad63d27bd5ca348b2ef9a7e2f44da289e436be6484ea28a1b033

Adapter versions: diagramGenerator=2, drawioFlowchart=1, drawioGantt=1, drawioSequence=1, drawioState=1

Full sidecar JSON: conduit-governed-live-data-cda4d617.json

Key modules & tables

src/server/data/bindings.ts — resolveBindingsForEntity + buildParams: the semantic→data join (entity/type → query + paramMap, entity_name/const/model sources)
src/server/data/executor.ts — executeNamedQuery, assertReadOnlySql (SELECT/WITH-only gate), recordDataCall: the governed gate that always writes provenance
src/server/data/adapters/* — getAdapter, wrapWithRowLimit, rewriteDollarParams; postgres.ts / mysql.ts / snowflake.ts are the only driver-touching modules
data_connectors (table) — AES-256-GCM encrypted creds, maxRows, timeoutMs, enabled, allowRawSql escape hatch
data_queries (table) — named parameterized read-only SQL templates ($1..$n) with DataParamSpec[] param specs (the 'domain components')
entity_bindings + data_calls (tables) — entity/type→query paramMap bindings; append-only [Dn] provenance (sqlText, paramsUsed, rowCount, durationMs, status ok/error/timeout/blocked, executedBy, run/conversation linkage)

Module reference: Conduit →Deep dive: Conduit →

6LensRetrieve & answer

The entry boundary resolves an explicit content-ACL context. Cache keys include that authorization scope and every stored answer must record its complete cited dependency set; serve revalidates authorization and hashes. On a miss, the same actor principals filter FTS/vector candidates and document-derived graph seeds before fusion, reranking, and context assembly. The pre-gate can clarify or abstain without generation; otherwise governed data calls add [Dn] provenance, grounded generation produces [Sn]/[Dn] claims, and groundedness/conflict signals feed the calibrated post-gate. Chat, structured REST, MCP, native agents, and evals propagate the same ACL contract.

Lens — retrieval and governed answers

Actor-scoped cache and retrieval feed graph fusion, ranking, policy, optional live data, and a cited structured result.

Downloads

Concepts

System diagrams
Lens — retrieval and governed answers
hybrid retrieval pipeline
Lens

Keywords

chunks · chunk_embeddings
canonical_relations · entities · mentions
clarify / abstain_with_pointers
policy_decisions · policy_calibrations
answer / answer_with_caveat / abstain
semantic cache
serve-time revalidate
miss · not cached
query understanding
hybrid retrieve()
effective query
graph fusion
tsvector + pgvector
1–2 hop expand
data layer
detected entities
[Dn] provenance
generate answer
groundedness judge
verbalized conf
calibrated confidence
fitted coeffs
record decision
store anchored answer
question + actor principals
exact → cosine → judge tier
complete deps + ACL scope
enforce hit · cached [Sn]/[Dn]
rewrite + sub-queries · 1 judge call
FTS ⋃ vector + ACL → RRF k=60
visible seed concepts only
token budget + MMR + front-load
entity detection + governance
draft / pending excluded
floor · entity ambiguity · LLM relevance
RankedChunk[] · [Sn] sources
execute governed bindings
grounded · boundary-tagged
context [Sn] + [Dn] · when includeData
chat · REST · MCP · agent · eval

Source and generation provenance

Status: current

Generated at: 2026-08-12T23:39:52.546Z

Source hash: 9f35ef38dbff5f310a1fec47756e94c8356435a7b325d0b9e03047997b952139

Metadata payload hash: 3bd8217e98283f0936085c3a1e2ea185d480e242c4210042ea030cbd41a45545

Canonical appearance

src/app/(docs)/docs/concepts/system-diagrams/page.tsx:333 route /docs/concepts/system-diagrams

All appearances

mirror — docs/architecture/system-diagrams.md:471
canonical — src/app/(docs)/docs/concepts/system-diagrams/page.tsx:333 route /docs/concepts/system-diagrams

Generation versions

App: eli-ai 0.1.0

Mermaid: 11.16.0 · Mermaid CLI: 11.16.0

Node: v26.3.1 · Yarn: 4.17.1

Renderer config hash: 68c10966fe84406ee626034d58bfabd555df9f65f691204b7c46db24038da101

Renderer theme hash: c80287a78d80ad63d27bd5ca348b2ef9a7e2f44da289e436be6484ea28a1b033

Adapter versions: diagramGenerator=2, drawioFlowchart=1, drawioGantt=1, drawioSequence=1, drawioState=1

Full sidecar JSON: lens-retrieval-and-governed-answers-9f35ef38.json

Key modules & tables

src/server/authz/content-acl.ts — principal context and SQL predicates
src/server/query/structured.ts — cache → retrieve → policy → data → generation → policy
src/server/retrieval/index.ts + pipeline.ts + graph-fusion.ts — actor-filtered FTS/vector/graph path
src/server/cache/semantic-cache.ts — ACL-scoped keys, all-or-none dependencies and serve validation
src/server/policy/router.ts + conversations/groundedness.ts — verdict and evidence checks
Tables: cache entries/dependencies, chunks/embeddings, canonical graph, data calls, policy decisions

Module reference: Lens →Deep dive: Lens →

7CrucibleEvaluate & measure

runEval pins the active golden set and applies an eval_configs row, then routes each item through the real Lens retrieve-and-answer pipeline. Target and judge snapshot ids are indexed foreign keys to immutable config_snapshots. Results carry deterministic error classes and roll up retrieval, answer-quality, abstention, concept, and significance measures. The two promotion paths remain distinct: /golden/promote accepts a supplied answer payload and does not consume a feedback row; authenticated POST /evals/feedback/promote validates a reviewed correction's run/result ownership, then atomically appends eval_feedback and creates its draft golden item, assertions, citations, and tags. A separate steward action must activate either draft, keeping the loop human-controlled.

Crucible — evaluation and quality control

Concept-anchored evaluation pins configuration lineage, measures the real Lens pipeline, and promotes reviewed corrections only as draft goldens.

Downloads

Concepts

System diagrams
Crucible — evaluation and quality control
Golden set — concept-anchored
Metrics — three measurable layers
Crucible

Keywords

judgments · run_metrics
drift.kb alert
golden_items · golden_assertions
golden_citations · golden_concepts
eval_runs · eval_results
deduped eval.gate.failed alerts
drift_snapshots + alerts
graded relevance 0–3
feedback row not input
answer payload →
Lens pipeline
each item × sample
retrieval IR
abstention + CRAG
append-only queue
validate · one TX
reviewed correction
named metric thresholds
rollout gate
BU × topic gap matrix
orphan rate · doc churn
Warrant curation
significant regression
thumbs · ratings · corrections
params applied + effective snapshot
catalog rows → forced-draft goldens
pin goldenSetHash · resolveEvalConfig
recall@k · nDCG@10 · MRR
faithfulness · relevancy · ctx · noise
truthfulness · risk–coverage · AURC
append feedback + create draft golden
paired bootstrap BCa + permutation
two-gate: CI ∌ 0 AND p &lt
latest-run accuracy overlay
steward review · re-certify
draft requires separate activation
re-author goldens · qrels
eval_configs
eval_feedback
eval_acceptance_gates

Source and generation provenance

Status: current

Generated at: 2026-08-12T23:37:46.765Z

Source hash: d72bba9d401008417665c3bb60732d8ef46aca2af6ed1fc5e40a3505fec4ed35

Metadata payload hash: 05824a2bcb18829d54c6e1eb17d34b1c90a59dbe67a8f26e7949d69b959742c4

Canonical appearance

src/app/(docs)/docs/concepts/system-diagrams/page.tsx:362 route /docs/concepts/system-diagrams

All appearances

mirror — docs/architecture/system-diagrams.md:526
canonical — src/app/(docs)/docs/concepts/system-diagrams/page.tsx:362 route /docs/concepts/system-diagrams

Generation versions

App: eli-ai 0.1.0

Mermaid: 11.16.0 · Mermaid CLI: 11.16.0

Node: v26.3.1 · Yarn: 4.17.1

Renderer config hash: 68c10966fe84406ee626034d58bfabd555df9f65f691204b7c46db24038da101

Renderer theme hash: c80287a78d80ad63d27bd5ca348b2ef9a7e2f44da289e436be6484ea28a1b033

Adapter versions: diagramGenerator=2, drawioFlowchart=1, drawioGantt=1, drawioSequence=1, drawioState=1

Full sidecar JSON: crucible-evaluation-and-quality-control-d72bba9d.json

Key modules & tables

src/server/evals/golden.ts — golden items, assertions, citations, concepts, qrels, and draft activation
src/server/evals/runner.ts — applied eval config, Lens execution, results, judgments, metrics, and pinned target/judge snapshots
drizzle/0033_eval_snapshot_fks.sql — indexed target/judge snapshot foreign keys with ON DELETE SET NULL
src/server/evals/metrics/ — retrieval, RAGAS, abstention/AURC, and significance metrics
src/server/evals/feedback-classify.ts + evals/feedback/promote route — deterministic classification and atomic reviewed-correction promotion
src/server/evals/drift.ts + src/server/coverage/index.ts — corpus drift, alerts, and coverage overlays

Module reference: Crucible →Deep dive: Crucible →

8PortsProgrammatic surfaces

Workspace keys bind one workspace; user keys resolve a live membership and capability set, with X-Workspace-Id when required. Both feed REST and the stateless eight-tool/two-resource MCP server under FORCE-RLS and an actor content-ACL context. Agent starts return 202 after an admitted queued outbox row; a stable agent-run identity plus boot/minute reconciliation repairs an interrupted broker handoff, and clients reattach through DB-backed SSE. HITL decisions atomically move suspended runs back to queued and the agent-resume worker idempotently claims the persisted cursor. All schedule kinds use scheduled_jobs desired state and stable-key reconciliation. Run events deliver through encrypted webhook secrets and the shared address-pinned, redirect-revalidating, bounded HTTP client.

Ports — REST, MCP, durable agents, and events

REST, MCP, schedules, durable runs/continuations, reattachment, and safe webhook delivery share one authenticated RLS control plane.

Downloads

Concepts

System diagrams
Key-gated surfaces · one RLS pipeline
Ports
Ports — REST, MCP, durable agents, and events
Durable run loop · enqueue → worker → reattach

Keywords

kb_* tools under RLS
reattach SSE
MCP clients · Claude · IDEs · agents
stable-key pg-boss reconcile
Grounded pipeline · withWorkspace RLS
encrypted HMAC · safe bounded HTTP
pg-boss agent-run / agent-resume
eli_sk_ / eli_uk_
agent_runs = queued
runner.ts durable loop
api_keys / user_api_keys
scheduled_jobs desired state
webhook_endpoints · webhook_deliveries
admitRun → executor.enqueue
run.completed / failed / suspended
Durable agents
worker start / resume
GET :id/stream
kb / data / reports
kb / data scope
allow / deny / edit
suspended → queued
HITL suspend · reaper
poll steps by cursor
REST clients · SDKs · playground
workspace + scopes + live capabilities + ACL
SHA-256 hash · scope grants
retrieval · graph · answerStructured
dispatchRunEvent → deliverAlert
query · documents · search · entities
hash lookup · requireScope
atomic decision (allow / deny / edit)
MCP
/api/v1
/api/mcp
/api/v1/runs
resolveAnyApiKey
Sn
Dn
REST

Source and generation provenance

Status: current

Generated at: 2026-08-12T23:40:51.634Z

Source hash: 6662b869be5ec7937a335aab8457df7bc68418af9ba412ffec22f39c4089f3e8

Metadata payload hash: e36785c18d608f4f8ced8005783fc9fd03f9f58b7806d23a66cfea6c31621238

Canonical appearance

src/app/(docs)/docs/concepts/system-diagrams/page.tsx:425 route /docs/concepts/system-diagrams

All appearances

mirror — docs/architecture/system-diagrams.md:624
canonical — src/app/(docs)/docs/concepts/system-diagrams/page.tsx:425 route /docs/concepts/system-diagrams

Generation versions

App: eli-ai 0.1.0

Mermaid: 11.16.0 · Mermaid CLI: 11.16.0

Node: v26.3.1 · Yarn: 4.17.1

Renderer config hash: 68c10966fe84406ee626034d58bfabd555df9f65f691204b7c46db24038da101

Renderer theme hash: c80287a78d80ad63d27bd5ca348b2ef9a7e2f44da289e436be6484ea28a1b033

Adapter versions: diagramGenerator=2, drawioFlowchart=1, drawioGantt=1, drawioSequence=1, drawioState=1

Full sidecar JSON: ports-rest-mcp-durable-agents-and-events-6662b869.json

Key modules & tables

src/server/apikeys + user-keys — hashed grants, workspace resolution and live capability narrowing
src/app/api/v1 + src/server/openapi/spec.ts — 56 documented REST operations
src/app/api/mcp + src/server/mcp/server.ts — ten tools and two resources
src/server/agents/executor.ts + starts.ts + continuations.ts + runner.ts + reattach.ts — durable start/resume/stream lifecycle
src/server/jobs/index.ts — 15 queues and scheduled_jobs desired-state reconciliation
src/server/alerts/webhooks.ts + network/safe-fetch.ts — encrypted signing secret and safe bounded delivery

Module reference: Ports →Deep dive: Ports →

See it end to end

These are the mechanics; the use-case guides walk a real workflow across the systems, and each demo workspace is a live example you can open and follow.

Searchable diagram catalog

Search every generated diagram metadata record and open route/source appearances plus five downloads (draw.io, SVG, PNG, Mermaid source, and JSON metadata).

Search terms

Type

Scope

Surface

234 of 234 diagrams

Page 1 of 10

Agent and API integration overview

Agent and API integration overview maps the principal capabilities, boundaries, and data paths in this architecture.

ArchivedFlowchartMarkdown

Concepts

End to end journeys

Keywords

notes_save RPC + RLS in SaaSHTTP MCP/ OpenAI-compatible APISSE events, git commit as agent

+17 more

Canonical: Canonical: Source
Mirrors: None

Downloads

Provenance and metadata

Canonical: Source (markdown, line 489)