RAG · retrieval-augmented generation
Serving pattern that retrieves external context and prepends it to the prompt, which makes traffic prefix-heavy and therefore highly sensitive to KV-cache reuse and routing.
Serving pattern that retrieves external context and prepends it to the prompt, which makes traffic prefix-heavy and therefore highly sensitive to KV-cache reuse and routing.