Skip to documentation
eli.aidocs

Atlas

Entity graph

eli.ai turns a pile of documents into a typed graph of entities (people, organizations, projects, topics, systems) connected by relations. Extracted facts carry provenance back to their source chunk; deterministic and manual knowledge carry their own explicit authority/provenance semantics. Three construction layers degrade gracefully and still provide useful local structure with a weak or absent LLM, without promising that every corpus becomes one connected component.

Ingestion pipeline

From save to the serve-time canonical_relations edge table.

Layer 1 — wikilinks (zero LLM)

On save, every [[Wikilink]] resolves deterministically against known entity names and aliases (NFKC-normalized exact match). A match writes a mention with its chunk and character offsets and a link_method recording how it linked; an unresolved target becomes a stub entity, flagged and hidden from client views. Your explicit links cost nothing and are the highest-precision signal in the system.

Layer 2 — co-occurrence (deterministic)

Wikilinks create entities but not edges, and a local model may extract nothing. So eli.ai derives a related_to edge between every pair of distinct canonical entities mentioned in the same document (capped per document to bound the pair count). Edge weight accumulates across the corpus, so entities that co-occur widely — shared systems, key people — naturally become hubs. It produces useful model-free local structure but does not guarantee that the whole corpus becomes one connected component. Each pair carries a relation_evidence row marked system:cooccurrence. Re-derivation first removes only that document's prior synthetic evidence, repairs orphaned machine edges, and writes the current pair set, so reruns and title changes are idempotent.

Layer 3 — LLM extraction (typed)

In the worker, each chunk is sent to the extraction model slot with a schema built from the workspace ontology. The model returns typed entities, typed relations, and the surface text of each mention — never character offsets, because models cannot count. Offsets are resolved afterward by normalization-tolerant, occurrence-ordered string search, never silently dropped.

  • Graceful degradation. Frontier models use native structured output (Output.object); OpenAI-compatible / local models fall back to schema-in-prompt with a JSON-repair round and ontology type-normalization.
  • Seeded ontology. Every workspace ships with a default ontology (Person, Organization, Project, Topic, Concept) so the type enum is never empty on first run. Structured output constrains the enum; fallback output normalizes to an allowed type.
  • Workspace-safe persistence. Extraction runs and every derived row carry workspace scope; there is no cross-workspace extraction-result cache.
  • Atomic latest-state replacement. New work enters the stately extract-v2 queue with one active and at most one queued intent per document. The worker snapshots the exact document, ordered chunks, and ontology, completes every provider call before graph mutation, then locks and revalidates those inputs and proves its run is still newest. One transaction wipes the prior model projection, applies every chunk, and completes the run. The last-good graph remains visible on provider failure or stale work.
  • Typed stale handling. Deleted documents and runs superseded by a newer owner are acknowledged; content/chunk or ontology changes enqueue a fresh latest-state intent. Upgraded workers drain the legacy extract queue, while producers use only v2.
  • Rejection tombstones. The extraction service honors persisted fact signatures. The current duplicate-review UI does not create these records; its dismiss action is client-local.

Non-destructive merges

"Acme Corp", "Acme", and "ACME Corporation" are the same entity. Merging them is a pointer, never a rewrite: the loser gets a merged_into pointer, its canonical_id updates in the same transaction, aliases repoint tagged with the merge_log id, and a log row records the actor and evidence.

Because every mention carries link provenance, undo is lossless: clearing the pointer, retracting merge-tagged aliases, and re-enqueuing the affected mentions restores the exact pre-merge graph. The duplicate-review inbox separately surfaces same-type, similar-name pairs for a human merge decision; dismissing one is not yet a durable rejection.

canonical_relations: the serve-time edge table

All three layers feed one materialized aggregate, canonical_relations. It is the onlyedge table read at serve time — by retrieval's graph channel, the graph explorer, and the agent's kb_graph_query tool. Raw mention-level rows are never queried live. Merges, splits, and re-extraction maintain it, so a graph traversal always sees canonical entities and de-duplicated edges. Document deletion purges its mentions and evidence before chunks disappear in the same tombstone transaction, retires only truly orphaned machine relations/entities, and rebuilds this projection. Manual/asserted relations and curated or clustered entities are preserved.

Provenance everywhere

Model-extracted and deterministic co-occurrence edges trace to relation_evidence; every extracted entity traces to its mentions. A human asserted relation may intentionally have no quote and is instead identified by its authority and mutation history. In the graph explorer, clicking a node opens a provenance panel that jumps to the source document at the exact chunk when one exists — and clicking an edge explains why the two entities are linked, with a deterministic strength score and its available evidence (see Relation explainability). The interface does not fabricate document evidence for manual knowledge.

Related