CAG (Cache-Augmented Generation)

A strategy in which a bounded knowledge base is loaded into the model's context window in advance and its KV cache is reused across queries. A new question consumes this precomputed state without a real-time document retrieval step. Here, CAG means Cache-Augmented Generation.

The design works best when the corpus fits in context, changes infrequently, and the runtime supports controlled cache reuse. It reduces retrieval latency and document-selection errors while introducing preload cost, invalidation work, stale-context risk, and possible attention dilution. Large or dynamic knowledge bases still favor RAG; hybrid architectures can cache stable knowledge and retrieve only what changes.