What Is RAG (Retrieval-Augmented Generation)?
Retrieval-Augmented Generation (RAG) is an AI architecture that combines a large language model with an external knowledge retrieval system, allowing the AI to access and cite specific documents or data sources rather than relying solely on its training data. Instead of generating answers purely from what the model learned during training, a RAG system first searches a knowledge base for relevant information, then provides that information to the language model as context for generating its response. This significantly reduces hallucination risk and improves factual accuracy, because the model can ground its answers in specific, verifiable source documents.
Retrieval-Augmented Generation (RAG) — an architectural pattern that combines a generative AI model with a retrieval system, so the model generates responses grounded in retrieved documents rather than only its parametric training data.
RAG is the most common pattern for deploying LLMs against enterprise knowledge — internal policies, technical documentation, customer history, regulatory text. It substantially reduces hallucination by grounding outputs in retrievable source documents, supports citations, and allows knowledge updates without retraining. RAG does not eliminate hallucination, and creates its own governance considerations around document access control, source quality, and prompt injection via retrieved content.
Source: NIST AI 600-1; OWASP LLM Top 10 (LLM01 Prompt Injection)
Why it matters for governance
RAG systems create specific governance obligations around the quality, currency, access controls, and legal compliance of the retrieval knowledge base. If the knowledge base contains outdated, biased, or incorrect information, the AI will generate confident-sounding outputs based on that flawed source material. RAG governance should address knowledge base maintenance and update procedures, access controls (who can add, modify, or delete source documents), data protection compliance for personal data in the knowledge base, and monitoring for retrieval quality and relevance.