Semantic ↔ data layer

Documents tell you what things mean; they cannot tell you what is true right now. eli.ai splits the difference explicitly: the knowledge graph extracted from your documents is the semantic layer, and it binds to a data layer — scoped, read-only external data sources queried through governed named queries — so live questions resolve against real data. Answers come back structured, with document citations ([Sn]) and data-call citations ([Dn]) side by side.

The architecture

The pipeline, end to end:

1. Documents → chunks → entities. Saving a document chunks and indexes it; extraction builds typed entities and concepts with full provenance — the entity graph. This is the semantic layer: it knows what an "Order Platform" is, who owns it, and which runbooks describe it.
2. Entities → bindings. An entity binding attaches a specific entity (or a whole entity type) to a domain component with a parameter mapping. The binding is the explainability record: it says why a data source is consulted for that entity.
3. Bindings → domain components → connectors. A domain component is a named, parameterized, read-only SQL query owned by a data connector — a scoped external source with encrypted credentials, a row cap, and a timeout.
4. Question → live lookup → dual-cited answer. When a question touches a bound entity, the kb_data tool discovers the binding and executes the component. Retrieved chunks become [Sn] sources; every executed lookup writes an append-only data_calls provenance row and becomes a [Dn] citation. The same answer shape flows to the UI, the structured query API, and the MCP server.

A worked example

Your vault documents an Order Platform — architecture notes, an incident runbook, an owner. Extraction has already made it a System entity in the graph. Now make it live:

1. Connect the ops database (read-only credentials, encrypted at rest):

{
  "name": "ops-postgres",
  "kind": "postgres",
  "config": { "host": "ops-replica.internal", "port": 5432, "database": "northwind_ops", "user": "eli_readonly", "sslMode": "require" },
  "maxRows": 200,
  "timeoutMs": 5000,
  "allowRawSql": false
}

2. Author a domain component — a named query a human wrote and governs. The model never writes this SQL; it only supplies validated parameters:

{
  "slug": "order-backlog",
  "title": "Order backlog by status",
  "description": "Current order counts and value by status over a recent window. Use for 'is anything stuck' questions about order processing.",
  "sqlTemplate": "SELECT status, count(*)::int AS orders, round(sum(total_usd))::int AS total_usd FROM orders WHERE updated_at >= now() - ($1::int * interval '1 day') GROUP BY status ORDER BY orders DESC",
  "params": [
    { "name": "days", "type": "number", "required": false, "default": 7, "description": "Look-back window in days" }
  ]
}

3. Bind the entity to the component. The binding declares that questions about the Order Platform entity may consult this component, and how its parameters are filled:

{
  "entityId": "01J… (Order Platform)",
  "queryId": "01J… (order-backlog)",
  "label": "Live order backlog",
  "paramMap": { "days": { "source": "const", "value": 7 } }
}

4. Ask a live question."Is anything stuck on the Order Platform right now?" The agent resolves the entity, discovers the binding via kb_data, executes the component, and answers from both layers:

The Order Platform currently has 14 orders in "stuck" status totaling
$8,120 [D1]. Per the incident runbook, stuck orders older than 24h should
be requeued via the fulfillment console [S1].

[S1] runbooks/order-platform-incidents.md · chunk 3        (semantic layer)
[D1] order-backlog(days=7) on ops-postgres · 4 rows · 41ms  (data layer)

[D1] is not a formatting nicety — it is backed by a persisted data_calls row recording the exact SQL text, the parameters used, row count, duration, status, and who executed it. Months later, the answer is still auditable.

The five pillars

The design holds itself to five best-practice pillars:

• Governed named queries over model-written SQL. The default surface is domain components: human-authored, human-gated SQL templates with typed parameter specs. The model supplies parameters, never statements. (A per-connector allowRawSql escape hatch exists for exploratory read-only SQL — off by default, owner opt-in, and still guarded.)
• Read-only by construction. Single-statement SELECT/WITH is enforced at save time andexecution time, on top of recommended read-only database credentials. Every call is bounded by the connector's maxRows and timeoutMs.
• Provenance for every call. Success, error, timeout, or blocked — every execution appends a data_calls row (SQL text, params, row count, duration, status, executor, run/conversation linkage). That row is the anchor behind every [Dn] citation.
• Explainability via bindings.A data source is never consulted "because the model felt like it" — it is consulted because a binding on the entity (or its type) says so, with a label a human wrote. The binding is inspectable on the entity's Live data panel.
• Workspace-scoped isolation. Connectors, components, bindings, and provenance are all tenant tables under FORCE-RLS. Credentials are AES-256-GCM encrypted, decrypted only at execution, and never returned by any API. External-connector SQL is network I/O and always runs outside tenant transactions.

Many engines, one contract

The data layer speaks to Postgres, MySQL, and Snowflake through one adapter contract: each engine gets a single adapter — the only code that touches its driver — implementing test-connection, schema discovery, and capped execution. Dialect differences stay inside the adapter: templates are always written with $1..$nplaceholders and a pure rewriter translates them to the driver's bind style ($n for Postgres, ? for MySQL and Snowflake), and every adapter returns identically normalized, JSON-safe rows. Everything described on this page — governed components, read-only enforcement, maxRows/timeoutMs caps, data_calls provenance, [Dn] citations — is engine-agnostic: a lookup against Snowflake leaves exactly the same audit trail as one against Postgres. Details per engine: Data connectors & domain components.

Natively available to other systems

The semantic↔data pipeline is not UI-only. Third parties call POST /api/v1/query and receive the full structured result — answer, claims, [Sn] sources, [Dn] data calls, entities, groundedness, usage — as JSON. External agents (Claude, or any MCP client) connect to the eli.ai MCP server and use kb_data_discover / kb_data_lookup alongside the KB search and graph tools.

The semantic layer also describes itself to machines: the workspace manifest (GET /api/v1/manifest, or the eli://workspace/manifest MCP resource) is a one-call snapshot of what the workspace knows — corpus counts, hub concepts by graph in-degree, and how much of it is human-vetted. That last dimension matters because concepts are not equally trustworthy: concept governance stamps every entity and relation with an authority tier and lifecycle status, so consumers of the semantic layer — chat, agents, the API, MCP clients — can tell a certified term of record from yesterday's machine extraction.