Overview
The Cache API stores frequently reused prompt prefixes so they can be referenced across chat-completion requests without re-sending the full text. This reduces latency and cost for workloads that repeat large system prompts or reference documents. The prompt cache is keyed by content hash, so identical prompts are deduplicated automatically.Base URL
The hosted Cloud API at
api.allternit.com serves only POST /v1/chat/completions and GET /v1/models under /v1. The endpoints on this page are served by the Allternit API data-plane node (http://127.0.0.1:8013/v1 locally); they are not routed on the public Cloud API origin. See Cloud API.Endpoints
Create a cached prompt
Request body
Example request
Example response
200. Otherwise the gateway returns 201.
List cached prompts
Retrieve a cached prompt
Delete a cached prompt
Using a cached prompt in completions
Reference a cached prompt in a chat-completion system message usingcache_id:
Error codes
Related pages
- Chat Completions — use cached prompts in messages
- Proxy — gateway middleware and routing