Chunking decides what a retrieval system can find. When docs are indexed for search or a chatbot, each page is split into chunks, each chunk is embedded, and questions are matched against chunks, not pages. A chunk that mixes three topics matches none of them well.
Good chunking follows the document's own structure: headings, sections, and paragraphs. That is why docs writing conventions like one idea per section and descriptive headings directly improve AI answer quality; they produce chunks that stand alone.
Chunk size is a trade-off. Chunks that are too small lose necessary context, such as which product version a step applies to; chunks that are too large dilute the embedding and waste the model's context budget.