Stop 15 · AI Agents & OrchestrationTopics · 6/9

Memory in Agents

In short

LLMs forget everything between API calls. If your assistant should remember that you're vegetarian, or what you asked five minutes ago, you have to build that memory yourself.

Mechanisms (conversation buffers, summarized history, vector-stored long-term memory) that let an agent retain context across a session or across sessions entirely.

The blob behind a shop counter holds a small notepad in one hand and a thick register in the other.

Think of it like this

The kirana uncle who knows you

The kirana uncle remembers today's list while you're at the counter. He also remembers, from years of visits, that your family buys Amul butter, never margarine. Agents need both: a short notepad for this chat and a long-term register for facts about you.

Watch it happen

Where an agent's memory lives

Agent memory

Short-term

Long-term

Recent turns

Running summary

User facts

Past episodes

1/5The model itself remembers nothing. Everything it 'knows' must be put into the prompt you send.

The big ideas

What you need to know

Context is the only memory

The model sees only what's in this request. Memory means choosing what to put back in.

Trim or summarise

Keep the last N turns word for word and summarise the rest. Long chats otherwise blow up cost and latency.

Store facts outside

Save stable facts in a database or vector store, keyed by user, and fetch them at the start of each turn.

Let users forget

Show users what is stored and let them delete it. Memory is personal data.

See it in code

Walk through the code

python

1MAX_TURNS = 62history = []   # short-term: recent turns3facts = {}     # long-term: saved facts about the user4 5def remember(role, text):6    history.append({"role": role, "content": text})7    if len(history) > MAX_TURNS:8        history.pop(0)  # forget the oldest turn9 10def build_messages(user_msg):11    profile = "; ".join(f"{k}: {v}" for k, v in facts.items())12    system = {"role": "system", "content": f"Known about user: {profile}"}13    return [system] + history + [{"role": "user", "content": user_msg}]14 15facts["diet"] = "vegetarian"16print(build_messages("Suggest a quick dinner"))

1/3Two kinds of memory: a list of recent turns and a dictionary of lasting facts.

Plain Python memory: a sliding window of turns plus a fact store. Frameworks add databases and summaries, but this is the core idea.

Save yourself the pain

Common mistakes

Oops

Sending the entire chat history on every call forever.

Do this instead

Cap the history and summarise older parts. Cost grows with every token you resend.

Oops

Saving everything the user says as a permanent fact.

Do this instead

Save only stable, useful facts, and let the user see and delete them.

Your mission

Give a chatbot a memory

~30 min

Quick check

Your agent forgets the user's name after a long chat, though it was said at the start. Most likely cause?

Go deeper

1Implement short-term memory (recent conversation) and compare it to a summarized-memory approach.
2Add a simple long-term memory backed by a vector store for facts across sessions.
3Learn the trade-off between memory size, cost, and relevance decay over time.