# Reading the token counts > Read the input and output token counts of every request in a short chat and in one prompt to an agent, see why a one-line question late in a conversation costs more than the same question at the start, and learn when to start a new conversation. Page: https://schubergphilis.github.io/ai-training/concepts/reading-token-counts/ License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/ A colleague spends a morning in one chat with an assistant. She asks what a kickoff email should contain, pastes her meeting notes, asks for a draft, pastes a budget sheet, and at the end asks for a subject line. The last question is one short line, and yet it cost about twice as much as the first one. In this lesson we read the token counts of every request in that chat and find out where the money went. Then we do the same for one prompt to an agent. You install and run nothing: every count on the page is in a table. The [lesson on what every token costs](https://schubergphilis.github.io/ai-training/concepts/what-tokens-cost/#what-a-token-costs) priced one call. The [lesson on one prompt and many requests](https://schubergphilis.github.io/ai-training/concepts/one-prompt-many-requests/#what-is-in-each-request) showed that each request holds the whole history so far. This lesson puts the two together. ## Two counts on every request When software sends a request to a model, the reply comes back with a short usage report, and that report holds two counts, the **input and output tokens** of the request (Messages, Claude Platform documentation). The vendor bills by these counts. The input count is everything the request sent to the model. It's the instructions, the tool descriptions if the assistant has tools, the history of the conversation and the new message. The tool descriptions are input tokens, and so are the tool results that come back (Pricing, Claude Platform documentation). The output count is everything the model wrote. That includes the visible answer and a request for a tool. A model that thinks before it answers writes its thinking as tokens too, and those are billed as output tokens (Manage costs effectively, Claude Code documentation). So the output count can be larger than the answer you see. Neither count is a count of words, because a token is a piece of text and not a word. The counts also include text you don't see, because the vendor turns the request into its own format before the model reads it (Messages, Claude Platform documentation). The two counts have different prices. On one vendor's price list an output token costs five times as much as an input token, on every model it lists (Pricing, Claude Platform documentation), and the example prices in the lesson on what every token costs follow the same pattern. So the same total can mean two different bills. Read the two counts separately, and price each one. Some usage reports split the input count further: tokens written to a cache, tokens read from it, and the rest. Together those parts are the whole input (Messages, Claude Platform documentation). The usage screen of one coding agent shows the same split for a session (Manage costs effectively, Claude Code documentation). The cache is the subject of the [lesson on caching and effort](https://schubergphilis.github.io/ai-training/concepts/caching-and-effort/). In this lesson every request is billed at the normal input price. #### Checkpoint: Where is most of the cost? A summary job sends a long report and asks for a short summary. Its usage report shows 40,000 input tokens and 500 output tokens, at $6 per million input tokens and $30 per million output tokens. Where does most of the cost of this request come from? - The summary, because each output token costs five times as much. - The input and the output cost about the same, because the prices make up for the counts. - The instructions, because they are sent first. - The report that was sent in, because the input count is eighty times the output. #### Checkpoint: Which count does it go in? Sort each part of a request into the count it adds to. Statements: - The instructions the assistant reads before every task - The descriptions of the tools the assistant can use - The thinking the model writes before its answer - A request for a tool in the model's reply - A tool result that goes back to the model - The model's own reply from two requests ago, sent again in the history Options: - Input tokens - Output tokens ## A short chat, request by request Here is the usage of the chat from the top of the page, one row per request. The counts are round numbers written for this lesson, and the prices are the example prices for a large model from the [lesson on what every token costs](https://schubergphilis.github.io/ai-training/concepts/what-tokens-cost/#what-a-token-costs): $6 per million input tokens and $30 per million output tokens. The left column says what was new in each request. The assistant's instructions are 500 tokens. ```text chat session request input output input $ output $ total $ 1. asks what a kickoff email holds 515 400 $0.0031 $0.0120 $0.0151 2. pastes meeting notes, asks for a draft 2,435 350 $0.0146 $0.0105 $0.0251 3. asks to make it shorter 2,795 200 $0.0168 $0.0060 $0.0228 4. pastes a budget sheet, asks to add it 5,015 250 $0.0301 $0.0075 $0.0376 5. asks for a subject line 5,277 15 $0.0317 $0.0004 $0.0321 total 16,037 1,215 $0.0962 $0.0365 $0.1327 ``` Read the table one column at a time. The output column goes up and down with the length of each answer. The input column only goes up. Request 1 sent the instructions and one question. Request 2 sent those again, plus the first answer, plus the pasted notes. Each request after that held everything before it, because the model keeps nothing between requests and the earlier messages are passed in again with each new one (Messages, Claude Platform documentation). This is **growing session cost**: every request in a conversation costs at least as much input as the one before it. The total row prices the summed counts, so a column added up by hand can differ from it by $0.0001. Now compare the first and the last row. Both questions are one short line. In request 1, most of the cost is output: 400 tokens of answer at the higher price. In request 5 the answer is 15 tokens, and almost all of the cost is input, the 5,277 tokens of conversation the subject line was sent with. The two pasted documents are in that input, and they were paid for again in every request after they arrived. The row below is the same subject-line question asked alone in a new chat, with only the instructions in front of it. The model then sees nothing of the email, so this row is for the price only. ```text question 5 alone, in a new chat 512 15 $0.0031 $0.0004 $0.0035 ``` The same question costs $0.0321 at the end of the chat and $0.0035 alone, about nine times as much. #### Checkpoint: Why did request 5 cost more? In the chat above, request 1 and request 5 were both a one-line question. Why did request 5 cost about twice as much as request 1? - Request 5 was sent with the whole conversation, both pasted documents included. - The price per token goes up the longer a chat stays open. - The answer to request 5 was longer than the answer to request 1. - The model had to search its memory of the chat for the earlier documents. ## One prompt to an agent A chat sends one request for each message you type, and an agent sends one for each step of its loop. One coding agent's cost guide explains that the agent calls the model again whenever tool results are ready, and that each of those calls holds the whole conversation so far (Manage costs effectively, Claude Code documentation). So one prompt to an agent can fill several rows of a usage report. Here is the toy run from the [lesson on one prompt and many requests](https://schubergphilis.github.io/ai-training/concepts/one-prompt-many-requests/#what-is-in-each-request): find the meeting note that mentions the June delivery, with a tool that lists a folder and a tool that reads a file. The agent's instructions and its tool descriptions are 1,400 tokens together. The counts are again written for this lesson. ```text one prompt to an agent request input output input $ output $ total $ 1. the prompt 1,430 40 $0.0086 $0.0012 $0.0098 2. result: 3 file names 1,530 40 $0.0092 $0.0012 $0.0104 3. result: the text of one note 3,970 120 $0.0238 $0.0036 $0.0274 total 6,930 200 $0.0416 $0.0060 $0.0476 ``` The person typed one line, and the report has three rows. The first two outputs are short, because a request for a tool is short. The input of request 3 is more than twice that of request 2, because the full note came back as a tool result and went into the history. If the agent had gone on to a fourth step, the note would have been in that request too. #### Checkpoint: How many requests? You give an agent one instruction. It uses eleven tools, one per reply, and then writes its answer. How many requests do you expect to see in the usage report for this instruction? - One request, because the person gave the agent one instruction. - Eleven requests, one for each tool the agent used. - Two requests, one before the tools ran and one after. - Twelve requests, one for each tool and one for the answer. ## Starting fresh The history is what makes a late request expensive, so the cheapest way to lower the cost is to drop the history. When you move to a task that has nothing to do with the conversation so far, start a new conversation. The old history adds to the cost of every new request and gives the new task nothing it needs. The vendor's cost advice for its coding agent gives the same rule for a switch to unrelated work, because text from the old task is paid for again with each new message (Manage costs effectively, Claude Code documentation). When you need to keep going on the same task, some tools can compact the conversation: they ask the model to write a short version of the history and continue from that. The requests after the summary are smaller. Making the summary is itself one large request, because the model has to read the whole history to write it. Opening a new conversation costs nothing by itself (Manage costs effectively, Claude Code documentation). #### Checkpoint: The next question is about something else You have spent the morning on one email in one chat. Now you want to ask about booking a train for next week, which has nothing to do with the email. What do you do? - Ask in the same chat, so the assistant has more context to work with. - Ask the assistant to forget the email first, then ask about the train. - Open a new chat and ask about the train there. - Ask the assistant to summarize the chat, then ask about the train. #### Pitfall: Blaming the long answers A team's assistant costs more than they expected, and the first guess is that its answers are too long. They add "keep answers short" to the instructions. The bill barely moves. The usage report shows why: the output counts were a few hundred tokens a request, and the input counts were tens of thousands, because a reference document was attached to every conversation and the conversations ran all day. The rule: read the input and the output counts before you guess, and look at what the input holds before you shorten the answers. ## Exercise Here is the usage report of another chat, about planning a team day. The counts are written for this lesson, at the same example prices. On paper or in a note, write for each request its input and output count, and say in one line what made the input grow. Then write two or three sentences on why request 4, a one-line question, cost more than request 1, and what you would do differently next time. Ten minutes is enough. ```text exercise: chat about a team day request input output input $ output $ total $ 1. asks for team day ideas 520 600 $0.0031 $0.0180 $0.0211 2. pastes a price list, asks which fits 4,120 300 $0.0247 $0.0090 $0.0337 3. asks for the invitation 4,435 250 $0.0266 $0.0075 $0.0341 4. asks if Friday is better 4,693 60 $0.0282 $0.0018 $0.0300 total 13,768 1,210 $0.0826 $0.0363 $0.1189 ``` A good result notes that the input grows on every row and the output does not. It names the pasted price list as the jump in request 2 and says the list was sent again with requests 3 and 4. It says that most of the cost of request 1 is its long answer, and most of the cost of request 4 is the history it was sent with. A good answer to the last part asks whether the Friday question needed the price list at all. To close, ask yourself: in your own last long chat, which pasted text did you pay for most often? Stretch: If a tool you use shows a usage screen with input and output counts, open it after your next long session. Find the request with the largest input and say what was in its history. ## Recap 1. Every request reports two counts. Input tokens are everything the request sent: instructions, tool descriptions, the history and the new message. Output tokens are everything the model wrote, thinking and tool requests included (Pricing, Claude Platform documentation) (Manage costs effectively, Claude Code documentation). 2. An output token costs several times an input token, so read the two counts separately and price each one (Pricing, Claude Platform documentation). 3. Each request in a conversation holds the whole history, so the input count only goes up. A short question late in a long chat can cost more than a long one at the start (Manage costs effectively, Claude Code documentation). 4. An agent sends a request for every step of its loop, and every tool result goes into the input of the requests after it. 5. Start a new conversation for an unrelated task. Compacting makes later requests smaller, and the summary is one large request of its own. #### Habit The next time a chat with an assistant moves on to a task that has nothing to do with the conversation so far, open a new chat for it. ## References - Messages, Claude Platform documentation, https://platform.claude.com/docs/en/api/messages - Pricing, Claude Platform documentation, https://platform.claude.com/docs/en/about-claude/pricing - Manage costs effectively, Claude Code documentation, https://code.claude.com/docs/en/costs