# What the model can see

> Measure a page of text in tokens and work out how many pages a context window holds, so a long chat that contradicts itself has an explanation.

Page: https://schubergphilis.github.io/ai-training/concepts/context-window/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

You have been working with a chat assistant on a report for two hours. In
your first message you said the report is for the finance team and must
stay under two pages. Now you ask for the closing section, and the answer
runs to four pages and addresses "the engineering team". The transcript
below is illustrative, written for this lesson rather than recorded from a
model, but the pattern is one you will meet.

#### Prompt (illustrative, not a recorded transcript)

```text
Write the closing section now. Same audience and length as we agreed.
```

#### Response

```text
Here is the closing section for the engineering team. I have kept the
technical detail you asked for, so it runs to about four pages. Let me know
if you want the deployment appendix as well.
```

The model did not forget in the way a person forgets. It never saw your
first message when it wrote this answer. In the previous lesson you learned
that a model reads tokens and predicts the next one from everything it has
seen so far. This lesson is about the limit on "everything it has seen so
far", the **context window**. In this lesson we measure a page of text in
tokens and work out how many pages a window holds, then watch a chat grow
past it.

## Count the tokens of a page

A context window is measured in tokens, so the first step is to get a
feel for how many tokens a page of text is. The counter below contains a
sample of meeting notes, about half a page. Press Count.

The first part of the report should read:

```text
characters: 460
words: 80
tokens (estimate): 115
```

The counter says "estimate" for a reason. It applies the common rule of
thumb for English, about four characters per token. A word of up to six
letters counts as one token and a longer word as one token per four
letters, with each punctuation mark counted as a token of its own. Real
tokenizers are more detailed than that, and every vendor has its own, so
the exact count for a model comes from that vendor. A call to the model
through the vendor's programming interface (API) returns the number of
tokens it used along with the answer, and vendors publish a token-counting
call or a web page that runs their tokenizer. For planning, the rule of
thumb is close enough: a full page of English text is a few hundred tokens.

## How a tokenizer splits text

The counter cuts "delivery" into two tokens by its four-letter rule. That
rule is a stand-in for what a real tokenizer does. A real tokenizer has a
fixed vocabulary of tens of thousands of chunks learned from training text,
and it cuts where that vocabulary happens to have a chunk. A common word
like "delivery" is in the vocabulary whole and costs one token, however
long it is. A rare word or a name is split into pieces instead, often at
places that have nothing to do with syllables or spelling. "Chidi" from the
sample is two tokens in current tokenizers, and a word in another language
may end up as one token per letter (How agents think: tokens and context, reasoning strategies, model choice, system prompts, Agent Engineer Course). The counter cannot know which
words are common, so it applies one rule to all of them.

Asking a model how many letters "strawberry" has goes wrong so often for
the same reason. The model receives the token for "strawberry" as one
number, and the letters are not in its input, so it predicts a count from
what it has seen written about the word, the same way it predicts any other
answer. Counting characters yourself, as the counter does, is the reliable
way.

## How many pages fit

The second part of the counter's report takes the token count from the
first part as the size of one page, and shows how many such pages fit in
windows of different sizes. For the sample it reads:

```text
one page: 115 tokens
8,000-token window: 69 pages
32,000-token window: 278 pages
200,000-token window: 1,739 pages
1,000,000-token window: 8,695 pages
```

The window sizes are round numbers that stand for the sizes on the market
in 2026. The 8,000 row is the size of a few years ago, kept for the
contrast. Small models that run on a laptop have windows at the low end
today, and the large vendor models offer 200,000 tokens or more (How agents think: tokens and context, reasoning strategies, model choice, system prompts, Agent Engineer Course).
At the top end the window holds a few thousand pages of text, which sounds
like more than you will ever need. The next section shows why it still runs
out.

A big window is not free to fill. The model processes every token in the
window for every response, which makes a fuller window cost more per answer
and take longer to answer. Nor does the model weigh every part of a full
window equally well: a fact in the middle of a long paste counts for less
in the prediction than the same fact at the top or at the bottom, as the
previous lesson noted under instruction dilution
(AI capabilities and limitations, Claude Academy).

## What the window holds

The window is not only your messages. For one response, the model sees all
of this at once, and the total must fit:

- The instructions the product or your team set up, which you may never
  see. In a chat product these are often a few pages by themselves.
- The conversation so far, with every message from you and every answer.
- Everything you pasted or attached, in full. A document you attach is
  turned into tokens like any other text.
- Results from tools the assistant used, such as a web search or a file
  that it read.
- The answer it is producing, token by token, since each new token is
  predicted from the whole window including the answer so far.

From this follows the one rule of the lesson: anything outside the window
does not exist for the model. Yesterday's chat, the file you did not paste,
and the company wiki are all invisible unless something puts them into the
window for this response (AI capabilities and limitations, Claude Academy). The
same rule explains why a model "remembers" nothing between conversations,
because the weights do not change when you type and so the window is the
only place a fact can be.

#### Checkpoint: What can the model see?

Buckets:

- In the window for this response
- Does not exist for the model

Items:

- The instructions the product set up before your first message
- The paragraph you pasted two messages ago in a chat that is still short
- The first half of the answer the model is writing right now
- The result of the web search the assistant just ran
- A chat you had with the same assistant last week, with memory features off
- A spreadsheet on your laptop that you mentioned by name
- A page of the company wiki that nobody pasted into the chat

## When the early facts fall out

Your chat with the assistant grows with every turn, and so does the text
sent to the model for each response. A long chat with pasted documents can
pass a 200,000-token window in an afternoon. Products handle the overflow
in different ways. Some silently drop the oldest turns, some replace them
with a short summary, and some refuse to continue and ask you to start a
new chat. In the first two cases you get the report chat from the top of
this lesson: the answers are still fluent, but a fact from the start of the
chat has left the window.

#### Pitfall: Scolding the model for forgetting

Two hours into the report chat, the assistant addresses the wrong team. You
reply: "I told you at the start this is for finance. Pay attention." The
next answer apologizes and then makes the same kind of mistake with the
page limit, because your correction named only the audience. The model
never saw your first message, so the scolding gave it nothing to work
with. The rule: when a long chat contradicts itself, restate the facts that
matter, in the message you are sending now. "This section is for the
finance team, two pages at most, plain language" puts the facts back in the
window, and the next answer has them.

Restating the facts is the fix in the moment. To prevent the problem, keep
the facts the model needs every time in one message you can paste again,
and start a fresh chat for a new piece of work instead of adding it to a
long one.

#### Checkpoint: Why did the answer change?

After two hours of chat, the assistant that had been writing for the
finance team in two pages suddenly writes four pages for the engineering
team. Which explanation fits what you know about how a model produces an
answer?

- The model changed its mind about the audience as the conversation went on.
- The first message was no longer among the tokens the model saw for this answer.
- The model was trained on more engineering documents than finance documents.
- Two hours is longer than the model can stay focused.

## Exercise

Copy about one page of your own writing, an email or a report section.
Replace the meeting notes in the counter above with it, and press Count.
The counter runs in your browser and sends your text nowhere.
Write down the three numbers: characters, estimated tokens, and how many
of your pages fit in a 200,000-token window. With these numbers you can judge whether a
document fits before you paste it, and after how many turns a chat is full.

A good result is one line like "my page is 3,100 characters, about 750
tokens, and 266 pages fit in 200,000 tokens." Then answer this: how many
pages of instructions, pasted documents and earlier turns would your usual
chat sessions need before the first message falls out of the window?

Stretch: Paste the same page into a chat assistant and ask it how many tokens the page is. Compare its answer with the counter, and decide which one you trust and why.

## Recap

1. A context window is the maximum number of tokens the model takes into
   account for one response, and a page of English text is a few hundred
   tokens (How agents think: tokens and context, reasoning strategies, model choice, system prompts, Agent Engineer Course).
2. The window holds the instructions, the conversation so far, anything
   pasted or attached, tool results and the answer being produced. Anything
   outside it does not exist for the model
   (AI capabilities and limitations, Claude Academy).
3. When a long chat contradicts itself, an early fact has left the window.
   Restate the fact in your next message instead of scolding the model.
4. A fuller window costs more per answer, because every token in it is
   processed for every response, and the model does not use every part of
   a full window equally well (AI capabilities and limitations, Claude Academy).

## References

- How agents think: tokens and context, reasoning strategies, model choice, system prompts, Agent Engineer Course, https://lsimons.github.io/agent-engineer-course/02-how-agents-think/
- AI capabilities and limitations, Claude Academy, https://academy.claude.com/courses/ai-capabilities-and-limitations