API Reference

For OpenAI/Anthropic SDK compatibility, see Gateway. This page covers the LLM Service Daemon’s (LSD’s) native endpoints, which add functions, variants, episodes, and feedback on top.

POST /inference

{
  "function_name": "generate_summary",
  "input": { "messages": [{ "role": "user", "content": "Summarize this." }] },
  "episode_id": null,
  "variant_name": null,
  "stream": false,
  "dryrun": false,
  "tags": { "team": "research" }
}

• Exactly one of function_name or model_name is required. model_name skips function config entirely and calls a model directly through LSD’s native request/response shape (functions/variants/episodes still apply, just with an implicit default function).
• Omit episode_id on the first call of a multi-turn interaction; LSD generates one and returns it in the response. Pass it back on subsequent calls to group them into the same episode. Never generate an episode ID yourself.
• Omit variant_name unless you’re deliberately pinning a variant for testing; otherwise LSD selects one per Experimentation.
• dryrun: true runs inference without persisting it.

POST /feedback

{
  "inference_id": "01957bbb-44a8-7490-bfe7-32f8ed2fc797",
  "metric_name": "thumbs_up",
  "value": true
}

• Exactly one of inference_id or episode_id is required, not both.
• metric_name is either a metric defined in config, or the reserved names comment / demonstration.
• value’s shape depends on the metric type: boolean, float, free-text comment, or a demonstration (a corrected output).

POST /batch_inference

Submits many inference requests as one job (same Params shape as /inference, as an array). Poll with:

GET /batch_inference/{batch_id}
GET /batch_inference/{batch_id}/inference/{inference_id}

Datasets

POST   /v1/datasets/{dataset_name}/datapoints              # create
PATCH  /v1/datasets/{dataset_name}/datapoints               # update
DELETE /v1/datasets/{dataset_name}/datapoints               # delete
POST   /v1/datasets/{dataset_name}/from_inferences           # build a dataset from stored inferences
POST   /v1/datasets/{dataset_name}/list_datapoints
POST   /v1/datasets/{dataset_name}/get_datapoints
DELETE /v1/datasets/{dataset_name}

Datasets back evaluations and optimization jobs.

OpenAI/Anthropic-compatible

POST /openai/v1/chat/completions
POST /openai/v1/embeddings
POST /anthropic/v1/messages

Standard OpenAI/Anthropic request and response shapes; "model" resolves through a model’s routing list (see Gateway). These don’t expose functions, variants, or episodes; use /inference for that.