What is RAG (Retrieval-Augmented Generation)?

A technique that feeds an LLM relevant documents at query time so answers are grounded in your data instead of only the model's training.

Retrieval-augmented generation solves a basic LLM problem: models only know what was in their training data, and they cannot see your private documents. RAG adds a retrieval step before generation. Your documents are split into chunks and converted into embeddings stored in a vector database. When a user asks a question, the system finds the most relevant chunks by semantic similarity and inserts them into the prompt, so the model answers from that material rather than memory alone. This grounds responses in current, private, or niche information, reduces hallucination, and allows citing sources. RAG quality depends heavily on the retrieval step: bad chunking or weak search means the model never sees the right passage. It complements rather than replaces fine-tuning; RAG supplies facts, fine-tuning shapes behavior.

Example

An internal HR chatbot uses RAG over the company policy handbook. When someone asks about parental leave, it retrieves the relevant policy section and answers from it, citing the page.

Why it matters

If you need an AI tool to answer from your own documents or up-to-date information, RAG is the standard approach, and it is a key feature to look for in products. Browse the AI tools directory or the model leaderboard to put it into practice.