Overview
The Cache API stores frequently reused prompt prefixes so they can be referenced across chat-completion requests without re-sending the full text. This reduces latency and cost for workloads that repeat large system prompts or reference documents. The prompt cache is keyed by content hash, so identical prompts are deduplicated automatically.Base URL
Endpoints
Create a cached prompt
Request body
Example request
Example response
200. Otherwise the gateway returns 201.
List cached prompts
Retrieve a cached prompt
Delete a cached prompt
Using a cached prompt in completions
Reference a cached prompt in a chat-completion system message usingcache_id:
Error codes
Related pages
- Chat Completions — use cached prompts in messages
- Proxy — gateway middleware and routing