RAG (retrieval-augmented generation)

Retrieval-augmented generation (RAG) is a technique where an AI model retrieves relevant passages from a knowledge base, such as product documentation, and uses them to generate a grounded answer.

For documentation, RAG works in two steps. First, the docs are split into chunks and indexed as embeddings; when a user asks a question, the system retrieves the most relevant passages. Second, a language model receives those passages as context and is instructed to answer from them. The model can still make errors.

The retrieval step gives a docs chatbot current documentation as context instead of relying on model training alone. That helps answers track documentation updates and makes source citations possible, but it does not eliminate model error.

RAG quality depends mostly on the retrieval side: how the docs are chunked, how well the index covers the site, and whether the system admits when nothing relevant was found. Biel.ai's chatbot is built on this pattern: it retrieves from your docs, cites sources, and is designed to decline unsupported questions.

Questions

How is RAG different from fine-tuning?

Fine-tuning changes a model through additional training. RAG retrieves source content for each question, so documentation updates need a refreshed index rather than model retraining.

Does RAG prevent hallucinations?

It can reduce unsupported answers, but it does not eliminate them. Retrieval grounds the model in real passages, but the system also needs to decline to answer when retrieval finds nothing relevant.

What does RAG stand for?

Retrieval-augmented generation. It means finding relevant source material, such as your documentation, before generating an answer.

RAG (retrieval-augmented generation)

Questions

Related reading