C4 · AI platform

The AI platform is a Vercel-AI-Gateway-routed model layer with a fixed four-tier registry (fast/smart/cheap/embed pinned in lib/ai/models.ts), a three-tier key resolver (SaaS workspace BYOK → local BYOK → gateway master key, lib/ai/resolver.ts), per-call usage metering with read-time pricing, seven native /api/ai/* feature routes, a persisted multi-turn RAG chat with an AI-SDK function-calling toolset over the vault, a daily digest pipeline (cron + manual), an OpenAI-compatible /v1 facade for external tools, AI template generation, and the full BYOK key lifecycle including pgcrypto encryption at rest and an offline KMS rotation script. An opt-in HIPAA lockdown (ELI_HIPAA=1) constrains all gateway routing to a BAA-eligible provider allowlist.

• Modes: local + SaaS (every route mode-dispatches; BYOK storage and auth diverge).
• Key dirs: lib/ai/, lib/chat/, app/api/ai/, app/v1/, app/api/cron/digest/, app/_actions/{ai-keys,ai-key-local,ai-gateway,chat,digest,templates-ai}.ts, components/ai/, scripts/rotate-ai-keys.mjs.
• Depends on: 01-core-data-storage (getStorage() adapters), 03-semantic-search (hybridSearch, chunkText, embedTexts), 05-agent-surface (PAT auth, lib/mcp/ratelimit), 09-platform-ops (env, Vercel cron, KMS key).
• Feeds: 02-authoring-ux (editor AI actions), 06-versioning-history (authorKind:'agent' version rows), 07-saas-collaboration (audit_log rows, workspace settings), 08-capture-templates (AI-generated templates).

Capabilities

System view — simplified

The AI platform is a thin, uniform stack: every caller — a UI component, an external OpenAI SDK client, or a cron job — hits a route that validates, authenticates, rate-limits, resolves a secret, and calls the Vercel AI Gateway with a tier-pinned model id and routing options. Standalone, this cluster is a complete "AI workbench over a markdown vault": with only a vault (or workspace) and one API key you get streamed drafting, summarize/transform, task extraction, map-reduce research, a persisted RAG chat with function-calling tools that can read and (opt-in) write notes, a daily digest note, and an OpenAI-compatible endpoint any third-party chat client can point at. All model traffic goes through the gateway — eli never talks to a provider API directly (the sole exception is the BYOK "Test" ping in lib/ai/test-key.ts), which is what makes failover, cost-sort routing, and the HIPAA provider lockdown single-point enforceable.

System view — detailed

Data flow — simplified

The load-bearing flow is the multi-turn ask-the-vault chat: the client sends only the new user turn plus a threadId; the server prepends stored history, retrieves context by hybrid search over the last user turn, compresses it via a fast-tier map step, and answers with a smart-tier reduce that can optionally call vault tools. Persistence is strictly owner-gated and never sinks the response on failure.

Data flow — detailed

Three sequences carry this cluster at runtime: the RAG chat with the tool-call loop, the /v1 facade serving an external OpenAI client, and the digest cron fan-out.

(a) Multi-turn RAG chat with tool loop (app/api/ai/chat/route.ts; the /chat/stream variant differs only in returning a UI-message stream and persisting in onFinish):

(b) /v1/chat/completions from an external OpenAI client (app/v1/chat/completions/route.ts):

(c) Digest cron run (app/api/cron/digest/route.ts → lib/ai/digest-runner.ts → lib/ai/digest.ts):

The manual path (generateDigestNowAction, app/_actions/digest.ts) reuses the same runDigest pipeline but is strictly session-scoped: it resolves the workspace from the eli_ws cookie plus membership, never uses the service role, never fans out, and tags generated_by: eli-manual. Digest browsing pages (/v/digest, /w/[workspace]/digest) discover digests by frontmatter type:digest, not path — see 08-capture-templates siblings and the trash/digest surfaces in 01-core-data-storage.

Works separately / works together

Standalone

With every other cluster reduced to its minimal interface (a storage adapter and a search function), the AI platform is a self-contained product: point one API key at it (AI_GATEWAY_API_KEY, a local BYOK key, or a workspace BYOK key) and you get drafting, summarizing, transforming, task extraction, research Q&A, a persisted chat that can read and write your notes with an audit trail, a daily digest note, and — via /v1 — the ability to use any off-the-shelf OpenAI client (Open WebUI, librechat, plain SDK scripts) as a frontend for your vault. Key lifecycle (encrypt, test, rotate, KMS re-encryption) and cost visibility (usage rows + read-time pricing) are built in, so the cluster is operable without any external billing or observability system. The HIPAA lockdown is a single env flag that hard-constrains every model call in the process.

Composed

• 03-semantic-search → chat/research/v1: hybridSearch(storage, query, {limit, workspaceId?}) drives RAG discovery in app/api/ai/chat/route.ts:178, chat/stream, research, and app/v1/chat/completions/route.ts:179; the search_notes tool wraps the same function (lib/ai/tools.ts:132-156); chunkText (3500/250) feeds the map steps; /v1/embeddings delegates to embedTexts(); MODELS.embed + EMBEDDING_DIMS=1536 are the canonical pins the embedding pipeline consumes.
• 01-core-data-storage ← tools/digest/extract-tasks: all reads/writes go through getStorage(); the digest cron constructs LocalAdapter.forVault / SupabaseAdapter.fromClient directly (app/api/cron/digest/route.ts:87,116); extract-tasks creates child notes under tasks/<source-slug>/.
• 06-versioning-history ← agent writes: update_note writes with {authorKind:'agent', agentName:'chat-tool'} (lib/ai/tools.ts:216), so every AI edit is a distinguishable version row.
• 07-saas-collaboration ↔ keys/audit/config: ai_keys (migration 0011), workspaces.ai_gateway_config (0012), and chat_threads/chat_messages (0014) live in the shared schema; recordAiAudit() lands ai.tool.* rows in the same audit_log the /settings/audit UI reads.
• 05-agent-surface ↔ /v1: /v1 routes reuse lib/auth/pat (extractBearer/parseBearer/verifyHash) against personal_access_tokens; rate limiting reuses lib/mcp/ratelimit; /v1/openapi.json advertises the whole AI surface to external agents; the same PAT credential covers /v1 chat + embeddings.
• 02-authoring-ux → feature routes: BlockNote slash-menu "AI continue"/"AI generate" call /api/ai/draft (components/editor/blocknote-editor.tsx:165-230), the toolbar TL;DR calls /api/ai/summarize, and AiTransformMenu calls /api/ai/transform.
• 08-capture-templates ← template generation: generateTemplateAction writes .eli/templates/<type>.md consumed by lib/templates/apply.ts and the template editor.
• 09-platform-ops → cron/env: vercel.json schedules /api/cron/digest at 0 13 * * *; CRON_SECRET, ELI_KMS_KEY, ELI_HIPAA, AI_GATEWAY_API_KEY are the operative env knobs; scripts/rotate-ai-keys.mjs is an ops runbook item.

Load-bearing details

• Model pins are code, not env: lib/ai/models.ts:31-40 deliberately rejects per-tier env overrides; the gateway routing config is the sanctioned override knob. EMBEDDING_DIMS=1536 at models.ts:62.
• HIPAA lockdown: ELI_HIPAA=1 forces only to [anthropic, azure-openai, google-vertex] (lib/ai/gateway-opts.ts:39,110) and throws on any explicit caller order naming a non-BAA provider (gateway-opts.ts:89-103); disallowPromptTraining: true on every call (gateway-opts.ts:112).
• Key resolution order is saas-byok → local-byok → gateway-default (lib/ai/resolver.ts:35-74). SaaS BYOK is only consulted when the caller passes BOTH workspaceId and provider — so routes without a provider body param (app/v1/chat/completions/route.ts:194-196, lib/ai/digest-runner.ts) can never use per-provider SaaS BYOK and silently fall through to the gateway master key.
• The resolved key rides as a per-call Authorization: Bearer header on the gateway request (app/api/ai/chat/route.ts:202); the source tag rides through responses and usage rows for telemetry.
• Chat map-reduce numbers: top-K default 8 (max 20), ≤24 chunks of 3500 chars / 250 overlap (chat/route.ts:213-222), fast-tier ≤100-word summaries, smart-tier reduce with cite-by-title instructions, tool loop capped by stepCountIs(6) (chat/route.ts:272,297).
• Chat persistence contract: only the turns received in the current call plus the final assistant reply are appended, and only when the thread exists AND is owned by the caller (local skips the owner check) (chat/route.ts:157-166,320-330); intermediate tool turns are never persisted; persistence failure only warns.
• Write tools are double-gated: useTools must be true AND allowWrites must be true for create_note/update_note to even appear in the toolset (lib/ai/tools.ts:164); the VaultChat UI mirrors this (allow-writes checkbox disabled unless tools checked, components/ai/vault-chat.tsx:283-305).
• SaaS agent writes produce best-effort audit_log rows ai.tool.create_note/ai.tool.update_note via service role (lib/ai/tools.ts:187,219, lib/ai/audit.ts); local mode's trail is the authorKind:'agent' version rows.
• /v1 local-mode auth quirk: when ELI_LOCAL_TOKEN is unset, ANY non-empty bearer is accepted (app/v1/chat/completions/route.ts:91-99, same in embeddings); SaaS requires an eli_pat_<prefix>_<secret> PAT with hash verification, revocation, and expiry checks (route.ts:102-122).
• /v1/chat/completions injects raw note bodies (first 4,000 chars per hit, route.ts:183-190) as RAG context, unlike /api/ai/chat which injects map-step summaries; tierFromModel maps eli/<tier> pins, ids containing mini/haiku/flash → fast, everything else → smart (route.ts:66-78).
• Pricing is an in-code table (lib/ai/pricing.ts:27-36: e.g. anthropic/claude-opus-4.6 at $15/$75 per MTok) with a $0 fallback for unknown models (pricing.ts:38-41) — pricing changes reprice history at read time without a backfill.
• Rate limits: 60/min and 10,000/day defaults (lib/mcp/ratelimit.ts:37-38), keyed user:<id> / pat:<id> / shared 'local' sentinel in local mode.
• Digest cron: vercel.json schedules 0 13 * * *; badAuth() refuses ALL requests when CRON_SECRET is unset (app/api/cron/digest/route.ts:46-51); SaaS fans out over ALL workspaces via SupabaseAdapter.fromClient on the service-role client (route.ts:106-132), while generateDigestNowAction is strictly session-scoped to the eli_ws cookie workspace and never service-role (app/_actions/digest.ts).
• Digest delivery is in-app only — a digests/YYYY-MM-DD.md note with type:digest frontmatter; no email/Resend integration exists; LLM Highlights failures degrade silently to the deterministic body (lib/ai/digest.ts).
• SaaS BYOK crypto: pgp_sym_encrypt via ai_keys_upsert RPC keyed by ELI_KMS_KEY (min 16 chars); plaintext never returned from any Server Action; the rotation script verifies round-trip decrypt per row and exits 3 on any failure (scripts/rotate-ai-keys.mjs).
• Anthropic key test is a 4-token claude-3-5-haiku-20241022 completion because Anthropic has no unauthenticated models-list endpoint (lib/ai/test-key.ts:51-54,109-140); Google receives the key as a ?key= query param.
• summarize persistToFrontmatter does a read-modify-write merge because updateNote replaces the frontmatter map wholesale (app/api/ai/summarize/route.ts:145-156).
• Local-mode chat threads/messages and ai_usage rows live in the per-vault SQLite index db — switching the active vault switches conversation history and usage stats (lib/chat/storage.ts:186-198, lib/ai/usage.ts:71-85).
• extract-tasks hard limits: source truncated at 50k chars, max 50 tasks, titles clipped to 200 chars, due dates must match YYYY-MM-DD (app/api/ai/extract-tasks/route.ts:58-88).
• Other input caps: draft prompt ≤20k chars, transform text ≤20k + hint ≤500, research maxChunks ≤50 (default 30) with ≤120-word map summaries, chat messages[] ≤50 — and the chat routes still accept the legacy single-shot {question} body (3–2,000 chars, synthesized into one user turn, app/api/ai/chat/route.ts:75,142).
• Vault tool caps: read_note truncates bodies at 30k chars with a [truncated] marker, list_notes limit ≤100 (default 20), search_notes limit ≤50 (default 10) (lib/ai/tools.ts:90,99,129).
• SaaS recordUsage silently drops rows without a workspaceId (lib/ai/usage.ts:55) — a metered call lacking workspace context produces no usage row at all.
• Chat storage encodings: thread lists are capped at 50 ordered by updated_at desc in both modes (lib/chat/storage.ts:52-53,204); local chat_messages store sources/tool_calls as JSON strings, and appends manually bump chat_threads.updated_at because SQLite has no touch trigger (storage.ts:288-295).
• SaaS renameChatThreadAction/deleteChatThreadAction perform no app-level ownership check — RLS (migration 0014 policies) is the only gate (app/_actions/chat.ts:97-127).
• Digest bullet formats diverge by mode: SaaS emits - [title](/w/<slug>/n/<path>) markdown links, local emits [[path|title]] wikilinks (lib/ai/digest.ts:72-82).
• SaaS BYOK accepts exactly five providers (openai/anthropic/google/xai/mistral) with plaintext 8–500 chars (app/_actions/ai-keys.ts:37-38,111).
• The /v1 SSE stream iterates result.textStream only — tool-call activity never appears in the emitted OpenAI chunk frames, just text deltas (app/v1/chat/completions/route.ts:287-297).

Gaps and partially wired

• Usage metering coverage is partial (verified by grep of recordUsage call sites): only /api/ai/draft, /continue, /summarize, /extract-tasks, /transform, the digest runner's LLM hook, and generateTemplateAction record usage. /api/ai/chat, /api/ai/chat/stream, /api/ai/research, /v1/chat/completions, and /v1/embeddings record nothing — the most expensive surfaces (smart-tier chat) are invisible in the usage cards.
• /api/ai/continue and /api/ai/extract-tasks have no UI consumer — the editor's "AI continue" slash command uses /api/ai/draft with the last 8k chars instead (components/editor/blocknote-editor.tsx:165-196). Both are API/agent-only, advertised via /v1/openapi.json.
• /api/ai/research has no UI consumer — API/agent-facing only, and it does not record usage.
• renameChatThread exists in lib/chat/storage.ts and as a server action (app/_actions/chat.ts) but has no UI consumer in VaultChat — threads keep their first-60-chars title forever.
• /api/ai/chat/stream has no first-party UI consumer — VaultChat calls the non-streaming /api/ai/chatand shows a "Thinking…" placeholder (components/ai/vault-chat.tsx:155); the streaming variant is API/agent-only today.
• Rate limiting is per-instance memory only — ELI_RATELIMIT_BACKEND=redis is env-selectable but throws at startup because lib/mcp/ratelimit-redis.ts never shipped (lib/mcp/ratelimit.ts:183-192); multi-instance SaaS deployments multiply the effective limits.
• /v1 local-token fail-open: with ELI_LOCAL_TOKEN unset, any non-empty bearer authenticates the local /v1 surface — a deliberate local-trust choice, but a footgun if a local instance is port-forwarded.
• SaaS BYOK unreachable from provider-less routes: /v1/chat/completions and the digest runner call resolveAiKey without a provider, so workspace BYOK keys are never used there; those calls always bill the gateway master key.
• Unknown model ids price at $0 in the usage UI rather than flagging — stale price-table entries silently understate costs (lib/ai/pricing.tsheader comment: "Rates as of 2026-Q2").
• No email delivery for digests— in-app note only; the digest pages' empty states point at a manual curl backfill against /api/cron/digest with CRON_SECRET.
• Local /v/digest has no inbound nav or command-palette link — URL-only surface (SaaS has a top-nav Digest link).
• Intermediate tool turns are never persisted to chat threads — replaying a thread loses the tool-call evidence; the only durable trail is version rows + SaaS audit_log.
• SaaS digest/chat pages resolve workspace chrome from the URL slug but notes from the eli_ws cookie workspace (shared platform quirk, see 07-saas-collaboration) — divergence shows one workspace's chrome with another's data.