Stop 22 · Interview PreparationTechnical · 5/8

RAG & Vector Database Design

In short

"How would you chunk these documents?" "Why is retrieval missing obvious answers?" "How does it scale to 10 million chunks?" RAG deep-dives are now common, and specific answers stand out.

A frequent deep-dive topic given how common RAG systems are in production — expect questions on chunking strategy, retrieval quality, and scaling a vector index.

The purple blob reaches into a neatly organised shelf of jars and sacks sorted into colour-coded bins.

Think of it like this

The well-organised kirana shelf

A good kirana uncle doesn't keep stock in one big heap. Similar things sit together: dals in one rack, spices in another. When you ask for moong dal, he goes straight to the right rack and picks from a few nearby packets. A vector index arranges chunks by meaning the same way.

Watch it happen

Design choices at each stage

Ingest

Chunk

Embed

Index

Retrieve

Rerank

1/6Extract text from PDFs and pages. Keep metadata like source, date and section for filtering and citations.

The big ideas

What you need to know

Chunking is a trade-off

Smaller chunks match precisely but lose context. Larger ones keep context but match loosely. Test a few sizes on real questions.

Hybrid search catches more

Vectors match meaning; keywords match exact terms like error codes and names. Together they miss less.

Metadata filters

Filtering by date, department or user permission keeps results relevant and stops people seeing documents they shouldn't.

Measure retrieval separately

Track recall@k: how often the right chunk is in the top k. Fix that before tuning prompts.

See it in code

Walk through the code

sql

1CREATE EXTENSION IF NOT EXISTS vector;2 3CREATE TABLE chunks (4  id        bigserial PRIMARY KEY,5  doc_id    text NOT NULL,6  content   text NOT NULL,7  embedding vector(384)              -- must match your embedding model's size8);9 10CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);11-- Top 5 nearest chunks to the question vector ($1), filtered by document12SELECT id, content, embedding <=> $1 AS distance13FROM chunks14WHERE doc_id = 'hr-policy'15ORDER BY embedding <=> $116LIMIT 5;

1/3Each chunk keeps its text, its source and its embedding. The vector size must match your model.

A vector store inside plain Postgres with the pgvector extension. Being able to sketch this shows you know a vector DB is a table plus an index, not magic.

Save yourself the pain

Common mistakes

Oops

Picking one chunk size and never testing others.

Do this instead

Build 30 test questions, try two or three chunk sizes, and compare recall@5 for each.

Oops

Relying on vector search alone for product codes, names and exact phrases.

Do this instead

Add keyword search like BM25 alongside vectors. Embeddings often blur exact identifiers.

Your mission

Chunk size experiment

~45 min

Quick check

Users search for error code "E-4021" and your RAG system keeps returning loosely related chunks. What's the best fix?

Go deeper

1Be ready to walk through your own RAG project's design decisions in detail.
2Practice explaining chunking, hybrid search and reranking trade-offs clearly.
3Prepare an answer for 'how would you evaluate whether your RAG system is actually good.'

More technical at this stop