Stop 14 · Retrieval-Augmented Generation & Vector DatabasesTopics · 3/9

Chunking Strategies

In short

You can't embed a 200-page PDF as one vector and expect precise answers. You cut it into chunks first. How you cut decides whether retrieval finds the right paragraph or a useless half-sentence.

Splitting long documents into smaller pieces before embedding them — chunk size and overlap directly affect retrieval quality, and getting this wrong silently degrades a RAG system.

The blob slices a long swiss roll into even pieces on a board, with slices overlapping on a plate.

Think of it like this

Slicing the swiss roll

Serve a whole swiss roll and nobody can eat it. Cut slices too thin and they crumble. Cut sensible slices, maybe with a little overlap so the swirl isn't lost, and every piece is complete. Chunking is slicing documents into pieces that each make sense alone.

Watch it happen

Fixed-size chunks with overlap

1/5A document of eight sentences. Each cell is one sentence.

The big ideas

What you need to know

Size is a trade-off

Small chunks are precise but lose context. Large chunks keep context but dilute the match. A few hundred tokens is a common start.

Overlap helps

A small overlap, like 10 to 20 percent, stops an answer being split across two chunks.

Respect structure

Split on headings, paragraphs or sections first. Recursive splitters fall back to sentences only when needed.

Carry metadata

Store the title, section and page with each chunk. It helps filtering and lets you cite sources.

See it in code

Walk through the code

python

1def chunk(text: str, size: int = 800, overlap: int = 150) -> list[str]:2    paras = [p.strip() for p in text.split("\n\n") if p.strip()]3    chunks, current = [], ""4    for p in paras:5        if len(current) + len(p) > size and current:6            chunks.append(current)7            current = current[-overlap:]          # carry a tail forward8        current += ("\n\n" if current else "") + p9    if current:10        chunks.append(current)11    return chunks

1/4Split on blank lines first, so paragraphs stay whole when possible.

A simple paragraph-aware chunker in characters. Libraries offer token-based and recursive splitters too.

Save yourself the pain

Common mistakes

Oops

Splitting every 500 characters, cutting sentences and tables in half.

Do this instead

Split on structure first: headings, paragraphs, then sentences. Keep tables and code blocks whole.

Oops

Choosing a chunk size once and never testing it.

Do this instead

Try two or three sizes on real questions and measure which retrieves the right chunk most often.

Your mission

Find the best chunk size

~35 min

Quick check

Why add overlap between chunks?

Go deeper

1Compare retrieval quality across different chunk sizes on the same document set.
2Learn fixed-size, sentence-aware and semantic chunking strategies.
3Practice adding overlap between chunks to avoid splitting relevant context awkwardly.