AI agent glossary
Chunking
Splitting long documents into smaller passages before embedding and indexing them, so retrieval can return just the relevant parts.
Retrieval systems rarely work on whole documents. A 40-page manual is split into chunks, each chunk is embedded and stored, and a search returns the few chunks most relevant to a question. How you split has a large effect on answer quality.
Simple approaches cut text into fixed-size pieces with some overlap. Better approaches follow the document's structure, splitting by headings, sections, or paragraphs so each chunk is a coherent unit. Chunks that are too small lose context; chunks that are too large dilute the match and waste the model's context.
Adding context to each chunk often helps, such as the document title and section heading, or a short generated note on where the chunk fits in the whole. Evaluate chunking choices on real questions rather than picking a size by default.
Learn more in the course
Related terms
Go from definitions to doing
Learn to use and build AI agents in 65 practical lessons.