# Where memory comes from

> Take the phrase "it remembered" apart into what was stored and what was read back, see how a long session drifts as its window fills, and pick summarize, start fresh or retrieve as the way out.

Page: https://schubergphilis.github.io/ai-training/concepts/memory/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

A colleague opens a new chat, and the assistant greets them by name and
asks about the report they worked on last week. "It remembered," they
say. An earlier lesson showed that a model has no memory of its own: the
weights don't change when you type, and the context window is the only
place a fact can be for the model. So something else happened here. A
fact was written down somewhere outside the window and put back in when
the new chat started. In this lesson we take the phrase "it remembered"
apart. Each time, the question is what was stored and what was read
back. The other end of the same subject is a long session that holds too
much, where the answers start to drift. You read a long transcript and
mark where the drift begins, then name the cure that would have helped.
You install and run nothing. By the end you can say where a memory is
stored and read back, recognize context rot in a session of your own,
and pick one of three ways out.

## Two places a fact can be

For a model, a fact is in one of two places. Either it is in the context
window right now, or it doesn't exist for this response. That rule came from the [lesson on the context
window](https://schubergphilis.github.io/ai-training/concepts/context-window/#what-the-window-holds). Memory doesn't
change the rule. It adds a second place where a fact can wait, and a
step that moves the fact back into the window.

**Short- and long-term memory** are the names for the two places
(Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course). Short-term memory is the context window itself: everything
said and seen in this session, the instructions, your messages, the
answers, the pasted documents, and the tool results. It grows with every
turn and is gone when the session ends. Long-term memory is anything
written down outside the window and read back later. A notes file the
assistant keeps, a summary of last week's chat, a database the agent
searches, or a note you keep and paste in again at the start of each
chat all count. Each of
these is text in a place the model can't see until something reads it
and puts it into the window.

The word "remembered" hides that step. When an assistant greets your
colleague by name, the name was in some earlier session, a product
feature wrote it to storage, and at the start of the new session the
product read the storage and placed the name in the window before the
first message. The model in the new session saw the name the way it sees
any other text in the window. A fresh session doesn't remember the last
one (AI capabilities and limitations, Claude Academy), and nothing about the
model changed between the two. That's also why a colleague with the memory feature switched off gets no
greeting: the same model, and an empty place where the note would have
been.

#### Checkpoint: Where is this fact?

Buckets:

- In the window for this session
- Stored outside and read back in

Items:

- The assistant repeats the page limit you gave it four messages ago
- A new chat opens and the assistant already knows your job title
- A coding agent follows a rule from a notes file in the project folder
- The assistant quotes a line from the document you pasted at the start of this chat
- An assistant answers a policy question from a passage a search found in the company wiki
- The assistant repeats the total it calculated two turns ago

## Where products keep the notes

Products give long-term memory different names, and they all come down
to text in storage, read back at a known moment. Two examples
show the pattern.

A coding agent such as Claude Code reads instruction files at the start
of every session. A project can hold a file that the team writes, with
build commands and conventions, and the agent also keeps notes of its
own, written during earlier sessions from your corrections and
preferences. Both are loaded into the context window at the start of
each conversation. Of the notes the agent writes itself, only the first
part of their index file is loaded. Those notes are on by default
(How Claude remembers your project, Claude Code documentation). The agent treats them as context, the
same as a message you type, so an instruction in such a file is a strong
default and never a guarantee. The [lesson on what tokens
cost](https://schubergphilis.github.io/ai-training/concepts/what-tokens-cost/#the-rule-that-got-buried) showed what
happens to rule seven in a long instruction file. Memory files are
instruction files, and the same limit applies.

Chat assistants offer memory features that save notes about you and
your work across chats. In the Claude chat assistant, memory saves notes
on topics such as your role and your projects while you chat. It is on
by default on the Free, Pro, and Max plans. On Team and Enterprise plans
it is off until an owner turns it on. The memory settings list every
saved topic, and you can read, change, or delete each one
(Use Claude's chat search and memory to build on previous context, Claude Help Center). Another product can save other things and
have another default, so read the settings page of the product you use
before you rely on the feature. In every case, the model in
the new chat sees whatever text the feature put back in the window. The
rest of the old chat isn't in it.

The third kind of long-term memory is the one from the [previous
lesson](https://schubergphilis.github.io/ai-training/concepts/retrieval/#search-first-then-answer). Retrieval is
memory that gets searched instead of loaded whole. The documents wait
outside the window, a search picks the passages that fit the question,
and only those are pasted in. Retrieval and memory differ in what they
hold, notes about you against shared documents (Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course). They share
the mechanism: storage outside the window, and a step that reads part of
it back in.

## When the window holds too much

Long-term memory covers one end of the problem, where the fact isn't in
the window. The other end is a window that holds too much. As a session
goes on, the window fills with old turns, tool output, pasted documents
and instructions that no longer apply. Nothing has fallen out yet. The
facts are all there, and the answers still get worse. The model follows
early instructions less reliably, loses a detail among the text around
it, contradicts something it said an hour ago, or asks for a fact you
gave it already. This decline is called **context rot** (Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course).

The [lesson on what tokens
cost](https://schubergphilis.github.io/ai-training/concepts/what-tokens-cost/#the-rule-that-got-buried) covered
instruction dilution, where one rule competes with the text after it in
one prompt or one file. Context rot is the session-level version. The
competition is between everything in the window, and it grows with every
turn, not only with the length of one file. A tool that returned a large
result you never looked at takes up as much of the window as a page you
wrote with care. Liu and colleagues found that models use a fact in the
middle of a long input less reliably than one at the start or the end
(Lost in the Middle: How Language Models Use Long Contexts, Transactions of the Association for Computational Linguistics 12 (2024), 157-173), and a long session puts most of the window in the middle.
So a long session fails a little at a
time. It drifts.

The signs are easy to name once you know them (Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course). The assistant
forgets a thing you said earlier in the same session. It contradicts an
earlier decision. It ignores a tool result from the start of the
session. It asks for information you already gave. Each of these, in a
session that has run for a while, is the cue to look at how full the
window is, before you look for anything else.

## Summarize, start fresh, or retrieve

The cure for context rot is to keep the window short and deliberate.
The source course on agent memory lists more ways to manage a full
window, for people who build agents (Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course). This lesson keeps the three a knowledge worker can
apply without tooling: summarize, start fresh, and retrieve. Each one
moves text out of the window or keeps it from getting in.

**Summarize.** Replace the old turns with a short summary of what
matters, and go on from the summary. Products do this for you under
names such as compaction. Claude Code, for example, runs a compaction
automatically as a session approaches the window's limit, and replaces
the conversation with a structured summary that keeps the instruction
files and the memory notes and drops the detail of the old turns
(Explore the context window, Claude Code documentation). A summary keeps the decisions and loses
the nuance, and the model that writes the summary can miss something
that mattered. So a summary is the cure for a long session whose history
matters, and it works best when you tell it what to keep.

**Start fresh.** Open a new session for the new piece of work, and bring
only the facts it needs, in one message you can paste again. You skip the
summary step and lose nothing to it, because you chose what to
carry over. This is the cure when the old session is about something
else, or when a summary would be as long as the facts you need.

**Retrieve.** Keep the documents outside the window and let a search
bring in only the passage the question needs. This is the retrieval from
the previous lesson, used as a cure. It is the right one when the
problem is a large document that was pasted whole and is needed one
paragraph at a time. The search becomes the weak point, as that lesson
showed, so it trades a full window for a search you have to check.

#### Checkpoint: The afternoon session drifts

The assistant that summarized the policy well at two o'clock now says the
policy allows something it forbids, and asks you which team the document
is for. You said so in your first message. What do you do?

- Tell the assistant it is wrong, and ask it to read the pasted policy again, more carefully this time.
- Ask for a summary of the decisions so far, then start a new chat with that summary and one section of the policy.
- Switch to a larger model in the same chat, because a stronger model keeps track of more text.
- Paste the full policy again at the end of the chat, so the assistant has a fresh copy to read.

#### Pitfall: The report that pasted the whole handbook

A team lead pastes a 90-page staff handbook into a chat and asks for a
one-page summary of the leave rules. The summary is good. They keep the
chat open all afternoon. Meeting notes get pasted in three times, and
forty more questions about other chapters get asked. By five o'clock the
assistant states a notice period from the wrong chapter and, when
corrected, apologizes and repeats a rule the handbook doesn't contain.
The handbook never left the window, and neither did anything else. The
rule: a large document pasted whole is the start of a long session, and
every later question about it reads a fuller window. Ask what you need from it. Then open a new chat with the
answer and only the section the next question is about. When the
document is one you ask about for weeks, a product that searches it
for you beats pasting it every time.

## Exercise

Below is a shortened transcript of a long session with an assistant.
The transcript is written for this lesson, and the assistant's lines are
illustrative rather than recorded. The user is drafting an internal
notice about a new expense process. Only the user's messages and the
first line of each answer are shown, and the turn numbers show how far
the session has come. Read it once from the top. Then, on paper or in a note, write down the number of the first turn where the answers start to
drift, the sign of context rot you see there, and which of summarize,
start fresh or retrieve would have helped most, with one line on why.
Ten minutes is enough.

```text
turn 1   User: Draft a notice for all staff about the new expense process. Audience is
              every employee, most of them not in finance. Plain language, one page at
              most, and every claim now needs a receipt photo in the app.
turn 1   Assistant: Here is a one-page draft in plain language ...
turn 2   User: Good. Here is the full 14-page finance procedure for context. [pasted]
turn 2   Assistant: Thanks, I have read the procedure. The draft still matches it ...
turn 5   User: Add a line on when claims are paid out.
turn 5   Assistant: Added: claims are paid with the next salary run ...
turn 9   User: Here are the notes of the finance meeting where this was decided. [pasted]
turn 9   Assistant: Noted. The meeting confirms the receipt rule ...
turn 14  User: Can you tighten the second paragraph?
turn 14  Assistant: Here is a tighter second paragraph ...
turn 18  User: Here are the notes from the IT meeting on the app rollout. [pasted]
turn 18  Assistant: Thanks. The app rollout dates are in these notes ...
turn 23  User: Update the notice with the rollout date.
turn 23  Assistant: Updated. Note that the procedure requires claims in the format
              set out in section 7.3, with cost center codes and approver initials ...
turn 24  User: That's for the finance team. The notice is for everyone, in plain language.
turn 24  Assistant: Apologies. Here is the notice again, now at two pages, covering the
              approval matrix from the procedure as well ...
turn 25  User: One page. Plain language. Every employee.
turn 25  Assistant: Understood. Which team is the notice for, and is there a length limit?
```

A good result names turn 23 as the first drift, because the answer
follows the pasted procedure's language for finance staff instead of
the audience and the one-page limit set at turn 1, and turn 25 as the
clearest sign, because the assistant asks for two facts the user gave in
the first message. Any of the three cures is defensible with a reason.
Summarize fits if the decisions of turns 5 through 23 matter and the
user wants to go on in the same product. Start fresh fits well here,
because a new chat needs only the turn 1 brief and the finished draft,
and leaves the 14-page procedure and two sets of notes behind.
Retrieve fits if the procedure gets asked about again and again, so
that a search brings in one section at a time instead of all fourteen
pages. If you were the user, which turn would you have stopped at, and
what would you have carried into the new chat?

Stretch: Take a long chat of your own, one that ran for an hour or more, and read it from the top. Mark the first answer that ignored something you had said. Which of the three cures would you apply, and at which message?

## Recap

1. A model has no memory of its own. Short-term memory is the context
   window for this session, and long-term memory is anything written
   outside the window and read back in. "It remembered" always means
   something was stored and re-read (Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course).
2. Products keep long-term memory as text in storage, loaded at a known
   moment: instruction files and the agent's own notes at the
   start of a coding session (How Claude remembers your project, Claude Code documentation), a chat product's
   memory notes (Use Claude's chat search and memory to build on previous context, Claude Help Center), or documents that a search
   brings in one passage at a time.
3. Context rot is the decline in a long session as the window fills
   with old turns, tool output, and stale instructions. The facts are
   still there, and the answers drift, contradict, and ask for what you
   already gave (Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course). A fact in the middle of a long context is
   used less reliably than one at the edges (Lost in the Middle: How Language Models Use Long Contexts, Transactions of the Association for Computational Linguistics 12 (2024), 157-173).
4. The cures are summarize, start fresh, and retrieve. Each moves text
   out of the window or keeps it from getting in, and the sign that you
   need one is an answer that ignores something you said in the same
   session.

## References

- Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course, https://lsimons.github.io/agent-engineer-course/05-memory-and-context/
- AI capabilities and limitations, Claude Academy, https://academy.claude.com/courses/ai-capabilities-and-limitations
- How Claude remembers your project, Claude Code documentation, https://code.claude.com/docs/en/memory
- Use Claude's chat search and memory to build on previous context, Claude Help Center, https://support.claude.com/en/articles/11817273-use-claude-s-chat-search-and-memory-to-build-on-previous-context
- Lost in the Middle: How Language Models Use Long Contexts, Transactions of the Association for Computational Linguistics 12 (2024), 157-173, https://aclanthology.org/2024.tacl-1.9/
- Explore the context window, Claude Code documentation, https://code.claude.com/docs/en/context-window