# Where an agent's memory lives

> Compare five places an agent can keep what it should know tomorrow, start with a notes file a person can read, see what changes when that file grows to ten thousand lines, and see how memory files and compaction keep an agent's context window small.

Page: https://schubergphilis.github.io/ai-training/building-agents/agent-memory/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

In this lesson we decide where an agent keeps what it should still know
tomorrow. [Retrieval as a tool the agent calls](https://schubergphilis.github.io/ai-training/building-agents/retrieval-as-a-tool/)
gave the agent a search over a handbook that somebody else wrote. Memory
is the store the agent writes itself: facts about a user, the state of a
long task, what it learned in the last session. The lesson compares five
places to keep those facts and argues for the simplest one as the first
version. The last section is about how a real agent platform uses memory
to keep the context window small. You pick a store for four uses at the end.

The examples run against a small fixture,
`site/examples/building-agents/agent-memory/memory.py` in the
repository, with the notes file `preferences.txt` next to it. Copy the
folder, and each step below is `python3 memory.py <step>`. No model is
called, so the numbers are the same on every machine.

## What the agent knows and what it can find again

During a response, a fact is either in the context window right now,
as part of the system prompt, the messages or this session's tool
results, or it is outside the window. A fact outside comes in only when
a step of the harness reads it and puts it there. The
[lesson on memory in the concepts course](https://schubergphilis.github.io/ai-training/concepts/memory/#two-places-a-fact-can-be)
called these short-term and long-term memory, and showed that "it
remembered" always means that something was stored and read back
(Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course).

For a builder, that read-back step is a design decision. The agent can
load a store whole at the start of every session, or it can look up the
part it needs when it needs it. Loading whole is simple and puts each fact
in the window. Looking up keeps the window small, but the agent has to
know that a fact exists and ask for it, and a lookup can miss. The rest
of this lesson is about which store makes that choice easy for each
kind of use.

Memory and retrieval use the same mechanism. The difference is what the
store holds. Retrieval reads shared documents that the agent didn't
write, and memory reads notes about one user or one task that the agent
or the user wrote (Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course). An agent often has both.

## Where a memory can be kept

**Memory storage choices** are the options for where long-term memory is
kept and how the agent finds a fact in it again. The course this topic
comes from names vector databases, knowledge graphs, and hybrids that
add a key-value store for session state and quick lookups. It also notes
that a retrieval step adds latency (Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course). The
table below is the list this course uses, with the case each one fits.

| Store                          | How the agent gets a fact back                               | Fits when                                                                                   |
| ------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------------------------------------- |
| A plain file read whole        | The harness loads the whole file at the start of the session | A few dozen facts that every session needs                                                  |
| Key-value notes                | A tool reads or writes one entry by its exact key            | Many small entries, each found by a name the agent already has, such as a user or task id   |
| A searchable index over text   | A tool searches by words or by meaning and returns passages  | A lot of free text, asked about in words nobody can list in advance                         |
| A database behind a tool       | A tool runs a query with filters, counts or sorting          | Structured records that are filtered, counted or combined                                   |
| A summary in the system prompt | Nothing to look up, because it is in every request           | The gist of a long history that every later turn needs, where losing the detail is accepted |

Each row trades something. The file is the easiest to read and fix, and
it costs its full size in every request. Key-value notes are cheap to
read one at a time, and useless when the agent doesn't know the key. A
similarity index can find passages in words that differ from the
document, which the keyword search in the
[retrieval lesson](https://schubergphilis.github.io/ai-training/building-agents/retrieval-as-a-tool/) couldn't do,
and it can return a passage that's close but wrong. For precise questions over
many records you need a database, and a database needs a schema and a
query tool that someone maintains. The summary is short in every request, and whatever
detail it dropped can't be read back.

#### Checkpoint: Which store fits each use?

Match each memory use to the store from the table that fits it.

Statements:

- Fifteen formatting rules that every reply must follow
- The progress note of each of 3,000 background jobs, read and updated by job id
- Four years of meeting notes, asked about by topic in the words of whoever asks
- Invoices, to answer how many unpaid invoices each customer has that are older than 90 days

Options:

- A plain file read whole
- Key-value notes
- A searchable index over text
- A database behind a tool
- A summary in the system prompt

## Start with a file a person can read

The first version of an agent's memory is a plain text file that the
harness reads whole at the start of each session. A person can open it,
see what the agent believes, and delete a wrong line. When the file is
in version control, each change the agent makes shows up in a diff. When the agent gets
something wrong because of a note, you can find the note. With the
other stores you have to build that yourself, and at the start that visibility
matters more than scale.

The worked case is a preferences file for one user. It holds twenty
facts, one per line, such as `date format: 25 September 2026` and
`email drafts: leave in drafts, never send`. The harness puts the file
into the system prompt, and the model reads it with every request.

#### Example: The preferences file read whole

Run `python3 memory.py whole` (the code is shown here) and compare what you see with the output.

```python
lines = load()
prompt_part = read_whole(lines)
print(f"preferences.txt: {len(lines)} lines")
print(f"read whole into the prompt: {len(prompt_part)} characters")
```

Output:

```text
preferences.txt: 20 lines
read whole into the prompt: 509 characters
```

The 509 characters are a short paragraph, and each request includes
them. A vector index over these twenty lines would need an embedding step, a
store and a search tool, and the agent would have to decide to search
before it knew that a preference applied. The file is always in the
window, so the model never has to remember to look.

Add a search tool when volume demands it. A word search over the file,
like the one below, is the simplest form of the searchable index row in
the table. Say the same file has collected a
line per project for months and now holds ten thousand lines. The
fixture builds that grown copy in memory and compares reading it whole
with a search tool that returns only the lines a question needs.

#### Example: The grown file, whole and searched

Run `python3 memory.py grown` (the code is shown here) and compare what you see with the output.

```python
lines = grown_notes(load())
prompt_part = read_whole(lines)
found = search_notes(lines, "project 0412")
print(f"grown notes: {len(lines)} lines")
print(f"read whole into the prompt: {len(prompt_part)} characters")
noun = "line" if len(found) == 1 else "lines"
print(f"search_notes('project 0412'): {len(found)} {noun}, {len(read_whole(found))} characters")
for line in found:
    print(f"  {line}")
```

Output:

```text
grown notes: 10000 lines
read whole into the prompt: 481545 characters
search_notes('project 0412'): 1 line, 49 characters
  project 0412: send the status update on Wednesday
```

The whole file is now 481545 characters in every request, and the one
line a task about project 0412 needs is 49 of them. At that size the first thing to
change is how the file is read. The agent stops loading it whole and
gets a tool that returns the lines a task needs. The file stays a file,
so a person can still open it. Split the twenty lines every session
needs into their own small file that's still loaded whole, and prune
the project lines that no longer apply. A vector index or a database is
the next step only if the search tool starts to miss.

#### Checkpoint: The preferences file has ten thousand lines

The agent's preferences file started at twenty lines and is read whole
at the start of each session. It now has ten thousand. What do you
change first?

- Move every line into a vector database first, and make each lookup a similarity search over embeddings.
- Keep loading it whole and move the agent to a model with a larger context window.
- Replace the file with a summary written by the model, and fold the summary into the system prompt.
- Stop loading it whole, keep the file, and give the agent a tool that returns only the lines a task needs.

## How memory keeps the context small

Memory also works in the other direction. A long agent task fills its
context window with tool results, and answers get worse as the window
fills. The concepts course called that decline context rot (Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course).
Agent platforms offer ways to move text out of the window and bring it
back when it is needed, and the Claude Platform documentation describes
three that work together.

The memory tool gives the model a directory of files that it can list,
read, create, edit, rename and delete. The tool runs on the client side.
The model sends a request such as `view /memories`, and code in your
application performs it on whatever storage you picked, for example one
folder per user or rows in a database (Memory tool, Claude Platform documentation). At the start of a task the model
looks in its memory folder and reads the notes there. While it works, it
adds new findings to those notes, and a later conversation picks them up
again. The page presents this as fetching text at the moment a task
calls for it, which leaves room in the window for the task at hand
(Memory tool, Claude Platform documentation). It is the plain file from the last section
with a tool in front of it. Among its safety advice, the page asks you
to put an upper limit on the size of each memory file and on the amount
of text one read hands back (Memory tool, Claude Platform documentation).

Context editing removes old tool results from the conversation once it
passes a size you configure, and puts a short placeholder where each
result was. When context editing is used with the memory tool, the model gets a
warning before results are cleared, so it can write what matters to a memory
file first (Context editing, Claude Platform documentation).

Compaction swaps the early part of a conversation for a summary, which
the model writes on the vendor's server, and a long task then fits in
the window again (Compaction overview, Claude Platform documentation). That's the last row of the table, applied to
the agent's own history. The memory tool page suggests turning on compaction and the memory tool
together for a long-running agent. The summary holds the size of the conversation down,
and anything the agent needs after the summary goes into a memory
file first (Memory tool, Claude Platform documentation).

#### Checkpoint: Where does the fact go?

A long agent task is close to its next compaction. The agent has found an exact
order number that it needs after the compaction. Where should that
number be kept?

- Leave it in the conversation, because the summary keeps every fact the model has seen.
- Write it to a memory file, because the files are outside the turns that the summary replaces.
- Repeat it in the next user message, because the most recent turn is never summarized.

#### Pitfall: A vector database for twenty facts

A team builds an assistant that should remember each user's
preferences. They set up an embedding model and a vector database, and
the agent searches it before every reply. Each user has twenty
preferences. The search sometimes returns the wrong three, the agent
ignores the date format it wasn't shown, and when a user reports it,
nobody can open the store to see what the agent believes. Twenty facts
fit in a file that's read whole. Start there, and add a search tool when a
measurement shows the file is too large to load.

## Exercise

Take four memory uses for an agent you might build. The agent keeps
twenty user preferences and can look up ten thousand support tickets.
It keeps a running task list for one long job, and it learns facts
during this session. For each one, pick a
store from the table in this lesson and write one line on why each of
the other stores costs more than it gives. Write it in a note of
about twenty lines, one for each pick and one for each store you passed
over. Doing this once gives you the questions to ask
before you add a database to an agent.

A good result picks by how the agent finds the fact. The preferences go
in a file read whole. The tickets are free text asked about in
unpredictable words, so they need a searchable index. The task list is a
small file the agent reads and rewrites, or a key-value entry per step.
Facts from this session are already in the context window, so the
question is which of them a later session needs. Those go to a memory
file, and the rest of a long history can be compacted into a summary. Each reason for
passing over a store names a cost, such as a lookup the agent must
remember to make or a schema someone has to maintain. Which of your four picks
would change first if the volume grew by a factor of a hundred?

Stretch: Pick one of the four uses and write down the size at which you would move it to the next store in the table, and what you would measure to know you had reached it.

## Recap

1. Memory is a store outside the context window and a step that reads it
   back. For each store, decide whether the agent loads it whole or
   looks up the part it needs (Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course).
2. The memory store follows from how the memory is used. Use a file for a few dozen
   facts, key-value notes for entries found by a known key, an index for
   free text in unknown words, a database for filters and counts, and a
   summary for the gist of a long history.
3. Start with a file a person can read and edit. When it grows too large
   to load whole, keep the file, give the agent a search tool, and split
   out the part every session needs.
4. Memory keeps the context small. A memory tool lets the model read
   files when it needs them, and compaction summarizes old turns. Write
   what the agent needs after the summary to a memory file first
   (Memory tool, Claude Platform documentation) (Compaction overview, Claude Platform documentation).

## References

- Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course, https://lsimons.github.io/agent-engineer-course/05-memory-and-context/
- Memory tool, Claude Platform documentation, https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-tool
- Context editing, Claude Platform documentation, https://platform.claude.com/docs/en/build-with-claude/context-editing
- Compaction overview, Claude Platform documentation, https://platform.claude.com/docs/en/build-with-claude/compaction