Skip to documentation
eli.aidocs

Platform

Multi-provider AI

eli.ai is provider-agnostic. Chat, extraction, judging, embeddings, and agents each resolve to a model slot, and every slot points at a provider you control — a frontier API, any OpenAI-compatible endpoint, or a local Ollama server. One choke point, resolveModel(workspace, slot), funnels every call through accounting, budgets, and the provider allowlist.

Supported providers

  • Frontier — OpenAI, Anthropic, Google. First-class providers, seeded automatically from OPENAI_API_KEY / ANTHROPIC_API_KEY / GOOGLE_GENERATIVE_AI_API_KEY, or added per-workspace in Settings.
  • Any OpenAI-compatible endpoint (openai_compatible) — add by base URL: vLLM, Together, LM Studio, a corporate gateway. Two manual capability toggles declare whether the endpoint supports tool-calling and JSON-schema output (there is no auto-probe — it misclassifies and rots when the endpoint restarts with different flags).
  • Ollama bridge — a one-click Settings preset that wires an openai_compatible provider at localhost:11434/v1 so everything runs offline. It is a local-development convenience, not a blessed production tier.

Model slots

Rather than hard-coding models, eli.ai binds five task slots per workspace. Each slot can point at a different provider and model:

  • chat and agent — a current top-tier model.
  • extraction and judge — a fast, cheap tier.
  • embeddings — the vector model backing retrieval.

First-boot seeding fills slots with pinned, dated defaults (per docs/agent-platform.md §12): Anthropic → claude-opus-4-8 (agent) / claude-sonnet-5 (chat) / claude-haiku-4-5 (extraction, judge); OpenAI → gpt-5 / gpt-5-mini / text-embedding-3-small; Google → gemini-2.5-pro / gemini-2.5-flash.

Never -latest

Slots pin dated model IDs, never -latestaliases. An eval can only compare two configs honestly if the model behind a slot doesn't silently change underneath it.

Workspace key beats env var

Credential precedence is inverted from the obvious default, deliberately. A workspace-stored key always wins; the env var is only the global fallback. This prevents two engagement-ending failures: a client workload silently billed to the consultant's personal key (a DPA violation), and a 2 a.m. scheduled job shipping a regulated client's transcripts to a forbidden provider.

  • Keys are stored AES-256-GCM under per-workspace data keys wrapped by a master key; the write API is write-only with a last-4 display.
  • A per-workspace provider allowlist is enforced inside the routing layer — only enabled providers resolve. A workspace pinned to a local model cannot have its content leave the box.
  • A "require workspace credential" flag makes a workspace hard-fail rather than fall back to the env key.

Cost accounting

Every model call — including embeddings, retries, and judge calls — is wrapped by accounting middleware (wrapLanguageModel / wrapEmbeddingModel), so nothing can bypass it. Each llm_calls row records tokens (with cache read/write split), cost as a 6-decimal string, latency, the credential source, and a config snapshot. A provisional row is finalized on finish or abort, so even a closed browser tab gets accounted.

one choke pointts
// Every call in the codebase goes through this — never a raw provider client.
const { model, modelId, params, credentialSource } =
  await resolveModel(workspaceId, "agent");
// model is already wrapped with accounting + budget + policy middleware.

When a slot is unconfigured, resolveModel throws a NotConfiguredError (status 409) that the UI maps to a setup screen and the API maps to { error: "no-model-configured" }.

Related