Latency Budgets

Latency budgets let organizations define per-model targets for end-to-end request latency and time-to-first-token (TTFT). The reporting endpoint compares actual percentiles from llm_usage_events against budgets and reports whether each percentile is within budget, exceeded, or missing data. All latency budget endpoints require the caller to be an organization owner or admin.

Base URL

For local development:

Endpoints

Budget fields

Use * as the model_id to define a global default budget.

List budgets

Response

Set a budget

Response

Delete a budget

Returns 204 No Content on success.

Latency report

window_hours defaults to 24 and is capped at 168 (7 days).

Response

Status values

Error codes