Retrieval-augmented generation (RAG): definition

Retrieval-augmented generation is a technique that fetches relevant passages from your own documents at question time and hands them to the model, so its answer is grounded in your data instead of its training.

An off-the-shelf model does not know your pricing, your policies, or last week’s ticket. RAG fixes that without retraining anything: when a question comes in, the system searches your knowledge base, pulls the few most relevant chunks, and puts them in front of the model as context. The model then answers from those chunks.

This is how most "chat with your docs" and internal-support tools actually work. Done well it cuts hallucination sharply and lets you update the model’s knowledge by editing a document, not by paying for a training run.

Related terms

Large language model (LLM): A large language model is a neural network trained on huge amounts of text to predict the next token, which lets it write, summarise, classify, and answer questions across almost any topic.
Embeddings: An embedding is a list of numbers that captures the meaning of a piece of text, so that passages about similar things sit close together and can be found by similarity rather than exact words.
Vector database: A vector database stores embeddings and finds the closest matches to a query vector quickly, which is what lets a RAG system retrieve the right passages from millions of documents in milliseconds.
Hallucination: A hallucination is when a model states something false or invented as if it were fact, because it predicts plausible-sounding text rather than looking anything up.