Skip to documentation
eli.aidocs

Ports

Approvals & feedback

Two human-in-the-loop surfaces: approvals pause a run before a risky tool executes, and feedback captures your judgment on answers so agents can improve. Both are durable, workspace-scoped records.

Approving gated tool calls

When an agent calls a tool flagged requiresApproval (all MCP tools in v1), the run suspends: its status becomes suspended and a run_approvals row is persisted with the tool name and exact input. Pending approvals collect at /w/<workspace>/approvals.

  1. Review the request

    The approval card shows the agent, the tool name, and the full input payload the model wants to send. You are approving this specific call with this specific input.

  2. Decide

    Choose allow or deny, optionally with a note. An allowed call executes and the run continues; a denied call returns an error result the model can react to.

  3. The run resumes

    Resumption reads the persisted cursor and picks up exactly where it paused — even hours later or across a process restart.

Durable by construction

Because the full run state is persisted at suspend, the human can take as long as they need. Approval decisions are also captured as feedback signal.
approve over the APIbash
curl -X POST https://your-host/api/v1/runs/$RUN_ID/approve \
  -H "Authorization: Bearer $ELI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"approvalId":"01J…","decision":"allow","note":"looks correct"}'

The in-app equivalent is POST /api/w/:workspaceId/approvals/:id; pending approvals list at GET /api/w/:workspaceId/approvals.

Capturing feedback

Feedback attaches to a run or a message and is the raw signal that drives the improvement loop. Five kinds:

  • thumb — { thumb: "up" | "down" }.
  • rating — { rating: 1..5 }.
  • correction — a free-text correction of what the answer got wrong.
  • edit — an edited final output.
  • citation_flag— a "wrong source" flag on a specific citation.
attach feedback to a runbash
curl -X POST https://your-host/api/v1/runs/$RUN_ID/feedback \
  -H "Authorization: Bearer $ELI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"kind":"rating","value":{"rating":4},"note":"missed the Q3 caveat"}'

From feedback to improvement

Feedback feeds three channels, all reusing the evals substrate:

  • Golden-set promotion. A corrected answer becomes a golden item (question + atomic assertions + required citations) after human approval, feeding automated eval runs.
  • Few-shot curation. High-rated exchanges become use-case-tagged few-shot examples, selected at prompt-assembly time.
  • Agent revision. Aggregated failure patterns inform a new immutable agent version, measured against the golden set before rollout.

The full loop is described in Evals & drift. The in-app feedback endpoint is POST /api/w/:workspaceId/feedback.