Glossary
/// Definition
Prompt caching
Prompt caching stores the prefix of a prompt at the provider so repeat requests skip recomputing it.
In depth
Anthropic and OpenAI both offer prompt caching. Long system prompts, RAG context and tool definitions can be cached on the provider side, cutting cost and latency dramatically for repeated queries.