Chunking
Chunking is the process of splitting documentation into small, self-contained passages before indexing, so a retrieval system can find and return the exact section that answers a question.
Chunking decides what a retrieval system can find. When docs are indexed for search or a chatbot, each page is split into chunks, each chunk is embedded, and questions are matched against chunks, not pages. A chunk that mixes three topics matches none of them well.
Good chunking follows the document's own structure: headings, sections, and paragraphs. That is why docs writing conventions like one idea per section and descriptive headings directly improve AI answer quality; they produce chunks that stand alone.
Chunk size is a trade-off. Chunks that are too small lose necessary context, such as which product version a step applies to; chunks that are too large dilute the embedding and waste the model's context budget.
Frequently asked questions
How big should a chunk be?
Most docs systems use a few hundred tokens, aligned to section boundaries, sometimes with overlap. Structure-aware splitting beats fixed-size splitting for documentation.
Can I influence how my docs are chunked?
Yes, by writing chunkable docs: a clear heading hierarchy, self-contained sections, and answers stated near the heading they belong to.