Skip to content
AI Training

Reading the token counts

A colleague spends a morning in one chat with an assistant. She asks what a kickoff email should contain, pastes her meeting notes, asks for a draft, pastes a budget sheet, and at the end asks for a subject line. The last question is one short line, and yet it cost about twice as much as the first one. In this lesson we read the token counts of every request in that chat and find out where the money went. Then we do the same for one prompt to an agent. You install and run nothing: every count on the page is in a table.

The lesson on what every token costs priced one call. The lesson on one prompt and many requests showed that each request holds the whole history so far. This lesson puts the two together.

Two counts on every request

Section titled “Two counts on every request”

When software sends a request to a model, the reply comes back with a short usage report, and that report holds two counts, the input and output tokens of the request [1]. The vendor bills by these counts.

The input count is everything the request sent to the model. It’s the instructions, the tool descriptions if the assistant has tools, the history of the conversation and the new message. The tool descriptions are input tokens, and so are the tool results that come back [2].

The output count is everything the model wrote. That includes the visible answer and a request for a tool. A model that thinks before it answers writes its thinking as tokens too, and those are billed as output tokens [3]. So the output count can be larger than the answer you see. Neither count is a count of words, because a token is a piece of text and not a word. The counts also include text you don’t see, because the vendor turns the request into its own format before the model reads it [1].

The two counts have different prices. On one vendor’s price list an output token costs five times as much as an input token, on every model it lists [2], and the example prices in the lesson on what every token costs follow the same pattern. So the same total can mean two different bills. Read the two counts separately, and price each one.

Some usage reports split the input count further: tokens written to a cache, tokens read from it, and the rest. Together those parts are the whole input [1]. The usage screen of one coding agent shows the same split for a session [3]. The cache is the subject of the lesson on caching and effort. In this lesson every request is billed at the normal input price.

Checkpoint · choice

Where is most of the cost?

Section titled “Where is most of the cost?”

A summary job sends a long report and asks for a short summary. Its usage report shows 40,000 input tokens and 500 output tokens, at $6 per million input tokens and $30 per million output tokens. Where does most of the cost of this request come from?

Checkpoint · match

Which count does it go in?

Section titled “Which count does it go in?”

Sort each part of a request into the count it adds to.

A short chat, request by request

Section titled “A short chat, request by request”

Here is the usage of the chat from the top of the page, one row per request. The counts are round numbers written for this lesson, and the prices are the example prices for a large model from the lesson on what every token costs: $6 per million input tokens and $30 per million output tokens. The left column says what was new in each request. The assistant’s instructions are 500 tokens.

chat session
request                                       input  output   input $  output $   total $
1. asks what a kickoff email holds              515     400   $0.0031   $0.0120   $0.0151
2. pastes meeting notes, asks for a draft     2,435     350   $0.0146   $0.0105   $0.0251
3. asks to make it shorter                    2,795     200   $0.0168   $0.0060   $0.0228
4. pastes a budget sheet, asks to add it      5,015     250   $0.0301   $0.0075   $0.0376
5. asks for a subject line                    5,277      15   $0.0317   $0.0004   $0.0321
total                                        16,037   1,215   $0.0962   $0.0365   $0.1327

Read the table one column at a time. The output column goes up and down with the length of each answer. The input column only goes up. Request 1 sent the instructions and one question. Request 2 sent those again, plus the first answer, plus the pasted notes. Each request after that held everything before it, because the model keeps nothing between requests and the earlier messages are passed in again with each new one [1]. This is growing session cost: every request in a conversation costs at least as much input as the one before it.

The total row prices the summed counts, so a column added up by hand can differ from it by $0.0001.

Now compare the first and the last row. Both questions are one short line. In request 1, most of the cost is output: 400 tokens of answer at the higher price. In request 5 the answer is 15 tokens, and almost all of the cost is input, the 5,277 tokens of conversation the subject line was sent with. The two pasted documents are in that input, and they were paid for again in every request after they arrived.

The row below is the same subject-line question asked alone in a new chat, with only the instructions in front of it. The model then sees nothing of the email, so this row is for the price only.

question 5 alone, in a new chat                 512      15   $0.0031   $0.0004   $0.0035

The same question costs $0.0321 at the end of the chat and $0.0035 alone, about nine times as much.

Checkpoint · choice

Why did request 5 cost more?

Section titled “Why did request 5 cost more?”

In the chat above, request 1 and request 5 were both a one-line question. Why did request 5 cost about twice as much as request 1?

One prompt to an agent

Section titled “One prompt to an agent”

A chat sends one request for each message you type, and an agent sends one for each step of its loop. One coding agent’s cost guide explains that the agent calls the model again whenever tool results are ready, and that each of those calls holds the whole conversation so far [3]. So one prompt to an agent can fill several rows of a usage report.

Here is the toy run from the lesson on one prompt and many requests: find the meeting note that mentions the June delivery, with a tool that lists a folder and a tool that reads a file. The agent’s instructions and its tool descriptions are 1,400 tokens together. The counts are again written for this lesson.

one prompt to an agent
request                                       input  output   input $  output $   total $
1. the prompt                                 1,430      40   $0.0086   $0.0012   $0.0098
2. result: 3 file names                       1,530      40   $0.0092   $0.0012   $0.0104
3. result: the text of one note               3,970     120   $0.0238   $0.0036   $0.0274
total                                         6,930     200   $0.0416   $0.0060   $0.0476

The person typed one line, and the report has three rows. The first two outputs are short, because a request for a tool is short. The input of request 3 is more than twice that of request 2, because the full note came back as a tool result and went into the history. If the agent had gone on to a fourth step, the note would have been in that request too.

Checkpoint · choice

How many requests?

Section titled “How many requests?”

You give an agent one instruction. It uses eleven tools, one per reply, and then writes its answer. How many requests do you expect to see in the usage report for this instruction?

Starting fresh

Section titled “Starting fresh”

The history is what makes a late request expensive, so the cheapest way to lower the cost is to drop the history. When you move to a task that has nothing to do with the conversation so far, start a new conversation. The old history adds to the cost of every new request and gives the new task nothing it needs. The vendor’s cost advice for its coding agent gives the same rule for a switch to unrelated work, because text from the old task is paid for again with each new message [3].

When you need to keep going on the same task, some tools can compact the conversation: they ask the model to write a short version of the history and continue from that. The requests after the summary are smaller. Making the summary is itself one large request, because the model has to read the whole history to write it. Opening a new conversation costs nothing by itself [3].

Checkpoint · scenario

The next question is about something else

Section titled “The next question is about something else”

You have spent the morning on one email in one chat. Now you want to ask about booking a train for next week, which has nothing to do with the email. What do you do?

Exercise

Here is the usage report of another chat, about planning a team day. The counts are written for this lesson, at the same example prices. On paper or in a note, write for each request its input and output count, and say in one line what made the input grow. Then write two or three sentences on why request 4, a one-line question, cost more than request 1, and what you would do differently next time. Ten minutes is enough.

exercise: chat about a team day
request                                       input  output   input $  output $   total $
1. asks for team day ideas                      520     600   $0.0031   $0.0180   $0.0211
2. pastes a price list, asks which fits       4,120     300   $0.0247   $0.0090   $0.0337
3. asks for the invitation                    4,435     250   $0.0266   $0.0075   $0.0341
4. asks if Friday is better                   4,693      60   $0.0282   $0.0018   $0.0300
total                                        13,768   1,210   $0.0826   $0.0363   $0.1189

A good result notes that the input grows on every row and the output does not. It names the pasted price list as the jump in request 2 and says the list was sent again with requests 3 and 4. It says that most of the cost of request 1 is its long answer, and most of the cost of request 4 is the history it was sent with. A good answer to the last part asks whether the Friday question needed the price list at all. To close, ask yourself: in your own last long chat, which pasted text did you pay for most often?

Stretch: If a tool you use shows a usage screen with input and output counts, open it after your next long session. Find the request with the largest input and say what was in its history.

Recap

  1. Every request reports two counts. Input tokens are everything the request sent: instructions, tool descriptions, the history and the new message. Output tokens are everything the model wrote, thinking and tool requests included [2] [3].
  2. An output token costs several times an input token, so read the two counts separately and price each one [2].
  3. Each request in a conversation holds the whole history, so the input count only goes up. A short question late in a long chat can cost more than a long one at the start [3].
  4. An agent sends a request for every step of its loop, and every tool result goes into the input of the requests after it.
  5. Start a new conversation for an unrelated task. Compacting makes later requests smaller, and the summary is one large request of its own.

You can now

  • Reads the token counts of a request and says where its cost comes from
  • Explains why a session costs more per request as it grows

References

Section titled “References”
  1. Anthropic. Messages. Claude Platform documentation. Reference. Claude docs messages
  2. Anthropic. Pricing. Claude Platform documentation. Reference. Claude docs pricing
  3. Anthropic. Manage costs effectively. Claude Code documentation. Reference. Claude Code costs