Capability clusters
C8 · Capture, templates & exchange
eli’s border crossing: three inbound doors (web clipper, phone share sheet, bulk import), two outbound doors (note download, vault zip), one template factory — all funneling through the same storage adapter as hand-written notes.
This cluster gets external content into the workspace (web-clipper bookmarklet + POST /api/clip, PWA share-target /share, bulk zip/JSON import) and out of it (single-note markdown download, whole-vault zip export), and it owns the note-template subsystem — a placeholder language ({{date}}, {{var}}, {{prompt}}), a parser + apply engine, an adapter-level applyTemplate surfaced as the apply_template MCP tool, a two-pane template editor, and AI-generated template skeletons. It also owns the PWA shell (manifest, service worker, offline fallback, iOS install guidance) that makes the share-target capture path installable on phones. Everything funnels through the same StorageAdapter contract as the rest of the platform (see C1 · Core data & storage), so a clip, an imported note, and a template-instantiated note are indistinguishable from hand-written notes once they land.
Modes: local + saas (bookmarklet/pairing and apply_template are local-only; install guidance page is saas) | Key dirs: lib/clipper/, lib/templates/, lib/pair/, app/api/{clip,import,export,pair}/, app/share/, app/_actions/templates*.ts, public/sw.js, app/manifest.ts | Depends on: storage adapters (C1), PAT + local-token auth (C5/C7), AI gateway (C4), frontmatter helpers (C1) | Feeds: FTS/semantic search (C3), version history (C6), authoring surfaces (C2), agent tools (C5)
Capabilities
| Capability | What it does | Entry points | Modes |
|---|---|---|---|
| Web clipper install page | Mints a fresh pairing token per page load, builds a javascript: bookmarklet, renders drag-to-install anchor + copy-source + pairing id for revocation (app/(local)/bookmarklet/page.tsx:24-47, components/clipper/bookmarklet-install.tsx:19-74); notFound() in saas | /bookmarklet | local |
| Bookmarklet runtime capture | Captures location.href, title, selection-or-body.innerText (50k cap) as pseudo-markdown plus outerHTML (200k cap); confirm() then POSTs to /api/clip with Bearer <token>; offers to open the created note (lib/clipper/bookmarklet-source.ts:35-47) | bookmarklet in any page → POST /api/clip | local |
POST /api/clip ingest | Bearer-authenticated JSON endpoint: saas = workspace PAT via service-role lookup, local = pairing token then MCP_LOCAL_TOKEN constant-time fallback; CORS * + OPTIONS preflight; 401/400/413/500 error mapping (app/api/clip/route.ts:44-145,163-235) | POST /api/clip, OPTIONS /api/clip | both |
| Clip handling core | Storage-agnostic handleClip: 60s idempotency window, title/path derivation to clipped/<slug>-<host>.md, collision-retry suffixes, data-URL attachment ingest (25MB cap, persisted saas-only) with markdown URL rewriting (lib/clipper/handler.ts:162-313) | called by /api/clip and /share action | both |
| Extension pairing | eli_pair_* token mint/verify/revoke/list; sha256 hash in ~/.eli/config.json#pairings[], plaintext shown once; approval page postMessages token to opener (lib/pair/extension.ts, app/api/pair/route.ts, components/pair-approval.tsx) | /pair?extensionId=…, POST /api/pair, GET /api/pair | local |
| PWA share-target capture | Manifest share_target GET-navigates /share?title=&text=&url=; editable confirm card whose server action composes markdown and feeds handleClip under the app session (no PAT), then redirects to the note (app/manifest.ts:75-83, app/share/page.tsx:20-168) | OS share sheet → /share | both |
| Bulk import | POST /api/import accepts a zip (JSZip walk of .md entries, 5000 cap) or {notes:[…]} JSON; upsert = getNote(path) hit → updateNote with agent 'import' attribution else createNote; returns per-path error list (app/api/import/route.ts:48-172) | POST /api/import (API/OpenAPI-only) | both |
| Single-note markdown export | Serializes frontmatter + body exactly as on disk, content-disposition attachment (app/api/export/note/[id]/route.ts:16-43) | GET /api/export/note/[id], inspector download button | both |
| Whole-vault zip export | Up to 5000 non-trashed notes at original paths + eli-export.json manifest so the archive round-trips through /api/import; streamed as eli-vault-YYYY-MM-DD.zip (app/api/export/vault/route.ts:22-74) | GET /api/export/vault, command palette | both |
| Template language (parser) | One-pass tokenizer → literal/date/var/prompt tokens; {{date:<fmt>}}, {{var:<name>}} (bare {{name}} sugar), {{prompt:<q>}}; friendly error on unmatched {{ (lib/templates/parser.ts:43-119) | parseTemplate() | both |
| Template apply engine | Renders whole source then re-parses frontmatter (unquoted {{date}} works in YAML); Day.js→date-fns token translation; unsupplied vars left in place + missingVars[]; _-prefixed frontmatter keys stripped; pluggable prompt runtimes (lib/templates/apply.ts:90-126, lib/templates/prompt-runtime.ts) | applyTemplate() | both |
apply_template MCP tool | {template, path, vars?} → storage.applyTemplate → reads <vault>/.eli/templates/<id>.md, runs engine, createNote; only LocalAdapter implements — Supabase/memory/json-file throw NotImplementedError (lib/mcp/tools/template.ts:32-74, lib/storage/local/writes.ts:451-480) | MCP tools apply_template, daily_note (companion) | local |
| Template management UI | Two-pane list/editor with Save/Delete/unsaved-guard; server actions do fs CRUD (local) or .eli/templates/-prefixed note CRUD (saas) (components/templates/template-editor.tsx:36-250, app/_actions/templates.ts:41-233) | /templates (local), /w/<slug>/settings/templates (saas) | both |
| AI template generation | generateTemplateAction resolves an AI key, calls generateText on the smart-tier model with a syntax-teaching system prompt, sanitizes output, persists to the templates path; usage recorded (app/_actions/templates-ai.ts:77-165) | editor “Generate with AI” button | both |
| PWA shell | Manifest (standalone, shortcuts Today/Search/Chat, share_target), production-only SW registration, eli-v2 service worker: network-first navigations with /offline.html fallback, cache-first static assets, /api/+/v1/+_actions always bypassed (app/manifest.ts, public/sw.js:22-118, components/pwa-register.tsx:12-27) | /manifest.webmanifest, /sw.js, /offline.html | both |
| iOS / desktop install guidance | Settings → About walks through Safari Share-Sheet → Add to Home Screen; root layout ships appleWebApp metadata + themeColor pair (app/(saas)/(app)/w/[workspace]/settings/about/page.tsx:18-79, app/layout.tsx:22-51) | /w/<slug>/settings/about | saas |
System view — simplified
C8 is the content airlock. Three inbound doors (clipper, share sheet, bulk import) and two outbound doors (note download, vault zip) all speak plain markdown + frontmatter, and one factory (templates) stamps out new notes from .eli/templates/*.md blueprints. Standalone — with search, AI, sync, and collaboration all turned off — this cluster is still a complete capture-and-portability toolkit: you can clip any web page into clipped/<slug>-<host>.md, share a link from your phone into an inbox note, batch-migrate a vault in or out as a lossless zip, and instantiate structured notes from templates with resolved dates and variables. Its only hard dependency is the storage adapter; everything else (AI generation, embeddings, agents) is additive.
System view — detailed
| Component | Role | Files |
|---|---|---|
| Clip handler core | Pure, storage-agnostic clip logic: body schema, 60s idempotency cache, title/path derivation, collision retry, data-URL attachment ingest with 25MB cap and markdown URL rewriting | lib/clipper/handler.ts |
| Clip API route | Thin Next wrapper: PAT (saas) / pairing-token-then-MCP_LOCAL_TOKEN (local) bearer auth, CORS + OPTIONS preflight, 401/400/413/500 mapping | app/api/clip/route.ts |
| Bookmarklet generator + install UI | Builds the javascript: bookmarklet embedding token + endpoint; local-only page mints a pairing token per load, drag-to-install + copy-source | lib/clipper/bookmarklet-source.ts, app/(local)/bookmarklet/page.tsx, components/clipper/bookmarklet-install.tsx |
| Extension pairing | Local credential mint/verify/revoke/list; eli_pair_* tokens hashed into ~/.eli/config.json; approval page postMessages plaintext to opener | lib/pair/extension.ts, app/(local)/pair/page.tsx, app/api/pair/route.ts, components/pair-approval.tsx |
| Share-target page | PWA share-sheet landing: confirm-and-edit card whose server action feeds handleClip using the app session (no PAT), then redirects | app/share/page.tsx |
| Import API | Bulk zip (JSZip) or JSON import, create-or-update upsert, 5000-entry cap, per-path error collection | app/api/import/route.ts |
| Export APIs | Single-note markdown download and vault zip (manifest eli-export.json, 5000-note cap) that round-trips through /api/import | app/api/export/note/[id]/route.ts, app/api/export/vault/route.ts |
| Template engine | Parser (literal/date/var/prompt tokens), applier (Day.js→date-fns translation, frontmatter merge with _-key protection, missingVars reporting), pluggable prompt runtimes | lib/templates/parser.ts, lib/templates/apply.ts, lib/templates/prompt-runtime.ts |
| Template storage + MCP surface | Adapter contract applyTemplate(templateId, {path, vars}); LocalAdapter implements against <vault>/.eli/templates/; supabase/memory/json-file throw NotImplementedError; exposed as apply_template | lib/storage/adapter.ts, lib/storage/local/writes.ts, lib/storage/supabase.ts, lib/mcp/tools/template.ts, lib/mcp/tools/daily.ts |
| Template management UI | Two-pane list/editor with Save/Delete/Generate-with-AI; server actions do fs CRUD (local) or .eli/templates/-prefixed note CRUD (saas) | components/templates/template-editor.tsx, app/_actions/templates.ts, app/_actions/templates-ai.ts, app/(local)/templates/page.tsx, app/(saas)/(app)/w/[workspace]/settings/templates/page.tsx |
| PWA shell | Manifest with shortcuts + GET share_target, production-only SW registration, stale-while-revalidate SW with offline navigation fallback, iOS install metadata + guidance | app/manifest.ts, components/pwa-register.tsx, public/sw.js, public/offline.html, app/layout.tsx, app/(saas)/(app)/w/[workspace]/settings/about/page.tsx |
| Discoverability surfaces | Command-palette “Export vault as zip”, inspector “Download as markdown”, OpenAPI 3.1 docs of export/import | components/command-palette.tsx, components/inspector/properties-panel.tsx, app/v1/openapi.json/route.ts |
Data flow — simplified
One click on the bookmarklet turns any web page into an inbox note: the captured text travels as pseudo-markdown over an authenticated cross-origin POST, handleClip derives a stable clipped/<slug>-<host>.md path, and the storage adapter persists file + SQLite row in one transaction. Full-text indexing is free — a SQLite trigger (lib/db/migrations/sqlite/0001_fts.sql:23-37) fires on the notes insert. The response’s viewUrl lets the bookmarklet offer to open the freshly clipped note.
Data flow — detailed
Sequence (a) — bookmarklet click → paired token → /api/clip → inbox note → index side effects:
Sequence (b) — create a template with AI, then apply it with prompt variables:
Mode divergence in one line each: clip auth is PAT-vs-pairing (app/api/clip/route.ts:163 vs :203), share auth is Supabase session + eli_ws cookie vs ELI_VAULT (app/share/page.tsx:138-156), template persistence is real files vs .eli/templates/-prefixed notes (app/_actions/templates.ts:67-77), and apply_template works only against LocalAdapter.
Works separately / works together
Standalone
With every other cluster switched off, C8 still delivers a working capture-and-portability layer on top of bare storage: install the bookmarklet (/bookmarklet), clip pages into clipped/ notes (attachment bodies are silently dropped in local mode — see Gaps), share URLs from a phone via the installed PWA (/share), migrate content in with POST /api/import, and get everything back out as a lossless zip (GET /api/export/vault → re-importable via the embedded eli-export.json manifest). Templates apply with dates and caller variables resolved using zero AI — the default prompt runtime degrades gracefully to (TODO: <question>) markers (lib/templates/apply.ts:128-132). The service worker keeps the app shell loadable offline with no server at all (public/sw.js:70-90).
Composed
- → C1 storage (Core data & storage): every capability funnels through
getStorage()/StorageAdapter—handleClipusescreateNote/putAttachment/updateNote, import/export uselistNotes/getNote/createNote/updateNote, templates use theapplyTemplatecontract (lib/storage/adapter.ts:173-176). Export and the template engine shareserializeFrontmatter/parseFrontmatter, guaranteeing round-trip fidelity. - → C3 search (Semantic search): clipped/imported notes become FTS-searchable instantly via the
notes_fts_aiinsert trigger (lib/db/migrations/sqlite/0001_fts.sql:23); semantic vectors arrive later viapnpm embed:backfill(package.json:20) or the debouncedscheduleEmbedfired by editor saves (app/_actions/notes.ts:96-105). - → C4 AI (AI platform):
generateTemplateActionconsumesmodelFor('smart'),resolveAiKey, gateway config andrecordUsage— the same plumbing as chat (app/_actions/templates-ai.ts:85-113). - → C5 agents (Agent surface):
apply_templateanddaily_noteMCP tools expose the engine to agents (lib/mcp/tools/template.ts,daily.ts:36-41explicitly points agents from empty daily notes toapply_template);/api/clipreuseslib/auth/patbearer parsing and theMCP_LOCAL_TOKENused by the MCP server;/v1/openapi.jsondocuments import/export for API consumers (app/v1/openapi.json/route.ts:293-300). - → C6 versioning (Versioning & history): import updates preserve version history and stamp
authorKind:'agent', agentName:'import'attribution (app/api/import/route.ts:144-172); every clip/create writes a versions row inside the same transaction (lib/storage/local/writes.ts:159-174). - → C7 saas (SaaS & collaboration): saas clip auth resolves workspace PATs via the service-role client (
app/api/clip/route.ts:163-198);/shareresolves the workspace from the Supabase session +eli_wscookie (app/share/page.tsx:138-156); import/export rely on RLS through the session-bound adapter. - → C2 authoring (Authoring UX): command palette hosts “Export vault as zip” (
components/command-palette.tsx:224-232), the inspector hosts “Download as markdown” (components/inspector/properties-panel.tsx:353-359), andnote:createdevents emitted by clip-created notes flow to the live UI via the events bus (lib/watch/events-bus.ts:24). - Internal composition:
/shareand/api/clipare two front doors to onehandleClip, sharing the sameglobalThisidempotency cache — a share-then-clip of the same URL within 60s dedupes (lib/clipper/handler.ts:137-152).
Load-bearing details
- No readability/article extraction exists: the bookmarklet sends
# title+ selection-or-document.body.innerText(50k cap) as “markdown”; Defuddle is referenced only in comments as a future extension pipeline (lib/clipper/bookmarklet-source.ts:42-43,lib/clipper/handler.ts:28-29);package.jsonhas no defuddle/readability dependency. - The bookmarklet embeds the plaintext pairing token in the
javascript:URL itself — the page warns “Treat the link like a password” (app/(local)/bookmarklet/page.tsx:66-68); every/bookmarkletreload mints a NEW pairing without revoking prior ones (page.tsx:15-19). - Clip idempotency is a 60s window keyed
workspaceKey::urlin aglobalThis-pinned Map (__ELI_CLIP_IDEMPOTENCY__, survives Turbopack per-bundle module isolation) — process-local, so multi-instance saas deployments would not dedupe across instances (lib/clipper/handler.ts:39-40,137-152). - Local clip auth composes the pairing id into the idempotency key (
<vault>::pair:<id>) so two extensions clipping the same URL get separate caches (app/api/clip/route.ts:209-212). /api/clipsetsAccess-Control-Allow-Origin: *plus an OPTIONS preflight handler; token compromise, not origin, is the stated security boundary (app/api/clip/route.ts:117-145)./api/importand/api/export/*have NO bearer auth — they rely entirely ongetStorage()(Supabase session + RLS in saas; single-tenant trust in local) (app/api/import/route.ts:26-28,app/api/export/note/[id]/route.ts:10-12).- The vault export zip embeds
eli-export.json {version:1,…}which the importer explicitly skips, making export→import a lossless round-trip of frontmatter + body at original paths; both sides cap at 5000 notes; entries are DEFLATE level 6 and an empty vault returns 404 (app/api/export/vault/route.ts:26-28,44-61,app/api/import/route.ts:88-97). - FTS indexing of clipped/imported notes is automatic: SQLite
AFTER INSERT/UPDATE/DELETEtriggers onnotesmaintainnotes_fts(lib/db/migrations/sqlite/0001_fts.sql:23-37) —persistRownever touches the FTS table directly (lib/storage/local/writes.ts:99-195). - Clips are NOT auto-embedded:
scheduleEmbed(30s debounce,lib/ai/auto-embed-queue.ts:33) is only invoked fromsaveNoteAction(app/_actions/notes.ts:96-105); clipped notes gain vectors viapnpm embed:backfill(package.json:20) or a later editor save. - The template engine renders placeholders across the ENTIRE source before parsing frontmatter, so
created: {{date:YYYY-MM-DD}}works unquoted in YAML;_-prefixed frontmatter keys from templates are always stripped as eli-managed (lib/templates/apply.ts:98-119). - Day.js→date-fns translation quotes literal
TAFTER token mapping so ISO timestamps ({{date:YYYY-MM-DDTHH:mm:ssZ}}) render correctly;Zmaps toxxx(±HH:mm offset) (lib/templates/apply.ts:66-88); malformed formats fall back to emitting the raw placeholder (apply.ts:140-146). - Template ids are validated with
/^[a-z0-9][a-z0-9-]*$/i(case-insensitive, ≤40 chars; bodies ≤40k) at get/save/delete boundaries (app/_actions/templates.ts:108,139-146,210-212); AI generation is stricter —/^[a-z][a-z0-9-]*$/i(letter-first) with hint ≤500 (app/_actions/templates-ai.ts:42-49) — all preventing path traversal through the.eli/templates/<id>.mdpath join. - Pairing tokens are
eli_pair_<26-char-ulid>_<base64url secret>with a 32-byterandomBytessecret (lib/pair/extension.ts:44-69,144); verification is constant-time (timingSafeEqualover sha256 hex) to prevent pairing-id enumeration, and theMCP_LOCAL_TOKENfallback comparison is also constant-time (lib/pair/extension.ts:133-139,app/api/clip/route.ts:238-245). - Clipped notes always land under
clipped/as<slug>-<hostname>.md(NFKD slug 80-char cap; hostnamewww.-stripped, 40-char cap); collision retry appends-YYYY-MM-DDthen-YYYY-MM-DD-<ms%1000>on duplicate-path errors (lib/clipper/handler.ts:341-361,363-389). - Attachments are capped at 25MB each (413
AttachmentTooLargeErrorwith filename + bytes), pre-validated BEFORE note creation to avoid orphan parents (lib/clipper/handler.ts:100-118,197-206). - Attachment persistence is saas-only:
LocalAdapter.putAttachmentthrowsNotImplementedError('Phase 5 (clipper)')(lib/storage/local.ts:299-300) andhandleClipswallows per-attachment failures (handler.ts:273-275), so local clips keep their original attachment URLs unrewritten; saas uploads to the Supabase Storageattachmentsbucket at<workspaceId>/<noteId>/<id>-<safeName>plus anattachmentsrow (lib/storage/supabase/attachments.ts:33-90). ClipBodySchemaexact caps:urlrequired and must parse as a URL,markdownrequired ≤2MB,html≤4MB,title≤500 chars,tags≤50 (each ≤80),attachments≤100 entries (lib/clipper/handler.ts:52-74).- The share-target uses method GET (not POST) so
/sharestays Server-Component renderable and survives hard reloads; URL-less shares get a synthetichttps://share.local/<slug>URL purely for path derivation; shared text is capped at 4000 chars (title 500) and empty shares — an OS double-fire quirk — bounce to/v(local) or/(app/manifest.ts:69-83,app/share/page.tsx:39-46,116,129-134). - The service worker deliberately never caches
/api/,/v1/,/_next/data/, or_actionspaths so RLS/auth always hit the live server; bumping the cache name (eli-v2) is the invalidation mechanism (public/sw.js:19,57-65). The prod-only registration gate checksNODE_ENV === 'production'— theNEXT_PUBLIC_PWAflag its comment mentions is honour-only, never read (components/pwa-register.tsx:8,15). - saas
saveTemplateActionparses template frontmatter with a line-basedkey: valueregex, not a YAML parser — nested YAML in templates will flatten to strings (app/_actions/templates.ts:180-189).
Gaps and partially wired
{{prompt:…}}AI resolution is a stub:aiPromptRuntimeunconditionally throws “requires the AI SDK (P5-AI-CHAT-01 carry-over)” and no dialog is registered fordialogPromptRuntime, so prompt placeholders resolve to(TODO: <question>)everywhere (lib/templates/prompt-runtime.ts:59-68,lib/templates/apply.ts:128-132).apply_templateis local-only:SupabaseAdapter.applyTemplatethrowsNotImplementedError('Phase 6 (templates)')— even though the saas template editor and AI generation happily store templates as.eli/templates/notes, no saas surface can apply them (lib/storage/supabase.ts:195-200vsapp/_actions/templates.ts:67-77).- Documented-but-missing
{{{{escape:parser.ts:30claims a handlebars-style escape, but no escaping code exists —{{{{actually parses as an empty-placeholder literal followed by braces. - No article extraction: raw
innerTextcapture only; the Defuddle/Readability pipeline exists solely in comments (lib/clipper/handler.ts:28-29). - HEIC→PNG conversion documented, not implemented: the sharp conversion is a carry-over noted in comments only (
lib/clipper/handler.ts:26-29). - Local clip attachments are dropped:
LocalAdapter.putAttachmentthrowsNotImplementedError('Phase 5 (clipper)')andhandleClipswallows per-attachment failures, so local clips enforce the 25MB cap but never persist attachment bodies — markdown keeps pointing at the origin URLs (lib/storage/local.ts:299-300,lib/clipper/handler.ts:273-275). ?format=mdon single-note export is documented but never read: the doc comment advertises the param yet the handler ignores the query string — output is always markdown (app/api/export/note/[id]/route.ts:7,16-43).- No saas bookmarklet page: saas users must manually paste a PAT into a hand-built bookmarklet;
/bookmarklet404s outside local mode (app/(local)/bookmarklet/page.tsx:20-25). - Pairing hygiene: each
/bookmarkletload mints a new pairing without revoking earlier ones, so~/.eli/config.json#pairings[]accretes stale entries; the approval page postMessages the plaintext token withtargetOrigin '*'(components/pair-approval.tsx:54-61). /api/importhas no UI: API/OpenAPI-only surface — no drag-and-drop or settings screen invokes it (app/v1/openapi.json/route.ts:293-300).- Process-local idempotency: the 60s clip dedupe cache does not span instances, so horizontally scaled saas deployments can double-clip (
lib/clipper/handler.ts:39-40). - saas template frontmatter round-trip is lossy for nested YAML (line-based
key: valueparser,app/_actions/templates.ts:180-189). - Clip-created notes are not embedded until a backfill run or a later editor save — a freshly clipped note is findable by FTS but invisible to pure semantic search in the interim (
app/_actions/notes.ts:96-105,lib/ai/auto-embed-queue.ts).
See also: the workspace overview for the platform overview, End-to-end journeys for cross-cluster walkthroughs that start with a clip, and Platform & ops for the env vars (MCP_LOCAL_TOKEN, ELI_VAULT) that gate local clip auth.