Data lineage

Lineage is the module slice that answers one question about any document, completely and in a single read: where did this come from, what knowledge did it produce, and what breaks if it changes? The other slices leave a trail — Intake records provenance, Atlas records which concepts a document mentions, Warrant records verification, Lens records which answers cited it — and Lineage stitches that scattered evidence into one DocumentLineage object. No LLM, pure SQL aggregates, deterministic ordering, every read tenant-scoped under row-level security.

The trail

Read left to right: upstream provenance and governance flow into the document; the document produces derived knowledge; that knowledge — plus the document itself — is what downstream consumers depend on.

The seven sections

A DocumentLineagehas seven parts, each sourced from a different slice's append-only record:

• document — identity and content fingerprint: path, version (bumped on every save), contentHash, and the created/updated actors.
• governance — Warrant's document verification tier (unverified · verified · certified · deprecated), who verified it and when, the next re-verify date, and reviewOverdue — true when a verified doc is past its SLA.
• origin — Intake provenance. kind is connector, vault, or upload; connector docs carry the source type (confluence · sharepoint · slack), external id, source version, last sync time, and up to ten recent sync runs.
• revisions — total plus up to 50 immutable summaries newest first. Each item carries version/change kind, sorted changed-field names, whether a recoverable content snapshot exists, activity, actor, note, and timestamp. Large content blobs never enter the lineage payload.
• audit — the append-only document.* event trail targeting this doc, newest first, capped at 50: create, update, verify, delete — with actor, timestamp, and meta.
• derived — Atlas knowledge the document produced: chunk count, the canonical concepts it contributed mentions to (top by mention count, capped at 50, each with authority and lifecycle), and the count of live relations whose evidence cites it.
• downstream — impact. Lens answers that cited it (citedByAnswers), the semantic-cache entries that depend on it, and impactedEntities — the live concepts grounded on it that a deprecation would touch.

The DocumentLineage shape

The full response type. Timestamps are ISO-8601 strings; nullable fields are explicitly | null. Arrays are deterministically ordered and capped so the payload is bounded.

type DocumentLineage = {
  document: {
    id: string; path: string; title: string; version: number;
    contentHash: string; createdAt: string; updatedAt: string;
    createdBy: string | null; updatedBy: string | null;
  };
  governance: {
    verification: "unverified" | "verified" | "certified" | "deprecated";
    verifiedBy: string | null; verifiedAt: string | null;
    nextVerifyAt: string | null;
    reviewOverdue: boolean;                       // verified/certified & past due
  };
  origin: {
    kind: "connector" | "vault" | "upload";
    sourceId: string | null;                      // connector_sources row (null = vault)
    sourceType: string | null;                    // confluence | sharepoint | slack | markdown_upload
    externalId: string | null;
    sourceVersion: string | null;
    lastSyncedAt: string | null;
    recentSyncRuns: {                             // up to 10, newest first
      id: string; kind: string;
      startedAt: string; finishedAt: string | null; error: string | null;
    }[];
  };
  revisions: {
    total: number;
    items: {                                      // newest first, cap 50; no content blobs
      id: string; version: number; changeKind: string;
      changedFields: string[]; hasContent: boolean;
      activity: string | null; changedBy: string | null;
      note: string | null; createdAt: string;
    }[];
  };
  audit: {                                        // document.* events, newest first, cap 50
    event: string; actor: string | null; at: string; meta: Record<string, unknown>;
  }[];
  derived: {
    chunkCount: number;
    entities: {                                   // top by mentionCount, cap 50
      id: string; name: string; type: string;
      authority: "machine_extracted" | "asserted" | "curated" | "certified";
      lifecycleStatus: "draft" | "pending_review" | "published" | "deprecated" | "superseded";
      mentionCount: number;
    }[];
    relationCount: number;                        // live relations whose evidence cites this doc
  };
  downstream: {
    citedByAnswers: number;                       // persisted answers whose [Sn] sources include this doc
    cachedAnswers: {                              // dependent cache entries, cap 50
      entryId: string; question: string; invalidated: boolean;
    }[];
    impactedEntities: number;                     // live entities grounded on it — the blast radius
  };
};

Read it over the API

One GET, scoped to the key's workspace. Accepts kb:read (preferred) or the legacy runs:read grant — the same read scopes as the manifest. The document ACL is checked before lineage is assembled, so an inaccessible restricted id is the same 404 as an unknown, cross-tenant, or soft-deleted id.

curl -s https://your-host/api/v1/documents/01KY10AAAAAAAAAAAAAAAAAAAA/lineage \
  -H "Authorization: Bearer $ELI_KEY" | jq '{origin: .origin.kind, concepts: (.derived.entities | length), citedBy: .downstream.citedByAnswers}'

{
  "document": {
    "id": "01KY10AAAAAAAAAAAAAAAAAAAA",
    "path": "ops/order-platform-runbook.md",
    "title": "Order Platform incident runbook",
    "version": 4,
    "contentHash": "9f2b1c…",
    "createdAt": "2026-06-02T14:20:00.000Z",
    "updatedAt": "2026-07-18T09:05:00.000Z",
    "createdBy": "usr_01KY0Z9M2E5Q4RS8T0V1WX2Y3Z",
    "updatedBy": "usr_01KY0Z9M2E5Q4RS8T0V1WX2Y3Z"
  },
  "governance": {
    "verification": "certified",
    "verifiedBy": "usr_01KY0Z9M2E5Q4RS8T0V1WX2Y3Z",
    "verifiedAt": "2026-07-01T00:00:00.000Z",
    "nextVerifyAt": "2026-12-28T00:00:00.000Z",
    "reviewOverdue": false
  },
  "origin": {
    "kind": "connector",
    "sourceId": "01KY0V000CONNECTOR00000001",
    "sourceType": "confluence",
    "externalId": "CONF-88213",
    "sourceVersion": "17",
    "lastSyncedAt": "2026-07-18T09:00:00.000Z",
    "recentSyncRuns": [
      { "id": "01KY18…", "kind": "incremental", "startedAt": "2026-07-18T09:00:00.000Z", "finishedAt": "2026-07-18T09:00:12.000Z", "error": null }
    ]
  },
  "revisions": {
    "total": 4,
    "items": [
      {
        "id": "01KY17…", "version": 4, "changeKind": "update",
        "changedFields": ["contentHash", "sourceVersion", "title"],
        "hasContent": true, "activity": "connector",
        "changedBy": null, "note": null,
        "createdAt": "2026-07-18T09:05:00.000Z"
      }
    ]
  },
  "audit": [
    { "event": "document.verify", "actor": "usr_01KY0Z9M…", "at": "2026-07-01T00:00:00.000Z", "meta": { "verification": "certified" } },
    { "event": "document.update", "actor": "usr_01KY0Z9M…", "at": "2026-07-18T09:05:00.000Z", "meta": { "version": 4 } }
  ],
  "derived": {
    "chunkCount": 12,
    "entities": [
      { "id": "01KY0ZW3…", "name": "Order Platform", "type": "system", "authority": "certified", "lifecycleStatus": "published", "mentionCount": 9 },
      { "id": "01KY0ZX8…", "name": "Fulfillment Console", "type": "system", "authority": "curated", "lifecycleStatus": "published", "mentionCount": 3 }
    ],
    "relationCount": 5
  },
  "downstream": {
    "citedByAnswers": 14,
    "cachedAnswers": [
      { "entryId": "01KY19…", "question": "how do we handle stuck orders?", "invalidated": false }
    ],
    "impactedEntities": 7
  }
}

Why it is trustworthy

• Tenant isolation is total. Every query runs under FORCE row-level security and also filters workspace_idexplicitly — workspace B can never read workspace A's lineage.
• It reads the append-only record. The audit trail and entity revisions are never mutated in place. The response exposes a bounded document-revision summary with the uncapped total, so clients can see both recent change shape and whether older history exists without transferring content snapshots.
• It is deterministic. No model call, stable ordering, bounded caps — the same document yields the same lineage every time, which is what makes it citable in a report or an audit.
• Downstream is live. citedByAnswers matches persisted answers whose [Sn] sources include this document via a jsonb containment query, and cachedAnswers reflects current cache dependencies — so impact is measured against what is actually deployed, not a snapshot.