What is document chunking in RAG?

Definition: Chunking is the process of splitting documents into passages before they are converted to embeddings and indexed, so a retrieval system can find and return the specific part of a document that answers a question.

How chunking works

Simple approaches split text every few hundred tokens with some overlap between chunks. Structure-aware approaches split on headings, paragraphs, tables or code blocks and attach metadata such as title, section and date. Some teams also add a short summary of the parent document to each chunk so it still makes sense out of context.

Why it matters

Chunks that are too large dilute the embedding and waste context; chunks that are too fine lose the surrounding meaning. Poor chunking is one of the most common reasons a RAG system retrieves the wrong passage, even with a strong model.

How teams use it

Teams pick a strategy per document type, such as contracts, support tickets or spreadsheets, test several sizes against real questions, and re-chunk when source formats change.

How Ryz helps

Ryz RAG engineers design ingestion and chunking pipelines for your documents, working on your team.

Related

• RAG development services
• RAG vs fine-tuning
• Retrieval-augmented generation
• Embeddings