Stop 15 · AI Agents & OrchestrationTopics · 3/9

LlamaIndex

In short

You have 300 PDFs of lecture notes and want to ask questions over them. LlamaIndex is built for exactly that: getting your own data into an LLM's answer.

A framework focused specifically on connecting LLMs to your data — indexing, retrieval and query engines — often preferred over LangChain for RAG-heavy applications.

The blob pulls one index card from a wooden card drawer and points at a matching book on a shelf.

Think of it like this

The college librarian's card catalogue

The librarian doesn't read every book when you ask a question. She has index cards that point to the right shelf and page. LlamaIndex builds those cards for your files, then hands only the right pages to the LLM.

Watch it happen

From a folder of notes to an answer

Load

Chunk

Embed

Index

Query

1/5SimpleDirectoryReader reads every file in notes/ into Document objects, with the file name as metadata.

The big ideas

What you need to know

Readers load your data

Readers turn PDFs, Word files, Notion pages or databases into Documents with text and metadata.

Nodes are chunks

Documents are split into nodes. Chunk size decides how much context each search hit brings back.

Query engines answer

A query engine retrieves the best nodes, then asks the LLM to answer from them. Chat engines add conversation memory.

Sources come back too

Every answer carries source_nodes, so you can show which file and paragraph it came from.

See it in code

Walk through the code

python

1# pip install llama-index  (uses OpenAI by default: set OPENAI_API_KEY)2from llama_index.core import VectorStoreIndex, SimpleDirectoryReader3 4docs = SimpleDirectoryReader("notes").load_data()5index = VectorStoreIndex.from_documents(docs)6index.storage_context.persist("storage")7 8engine = index.as_query_engine(similarity_top_k=3)9answer = engine.query("What did the professor say about deadlocks?")10print(answer)11print(answer.source_nodes[0].node.metadata["file_name"])

1/4Read every file in the notes folder into Document objects.

RAG over a folder in a few lines. The last line shows which file the best match came from.

Save yourself the pain

Common mistakes

Oops

Rebuilding and re-embedding the whole index on every app start.

Do this instead

Persist the index and load it with load_index_from_storage. Only re-embed files that changed.

Oops

Trusting the default chunk size for every kind of document.

Do this instead

Try a few chunk sizes on real questions. Dense textbooks and short FAQs need different settings.

Your mission

Ask your own notes

~30 min

Quick check

Your LlamaIndex app gives a wrong answer even though the right PDF is in the folder. What should you check first?

Go deeper

1Build a basic RAG pipeline using LlamaIndex's index and query engine abstractions.
2Compare LlamaIndex's RAG-focused API against a hand-rolled RAG pipeline you built earlier.
3Learn when LlamaIndex's higher-level abstractions save real time vs. add complexity.