# Reasons about what a model request costs and why

> Competency concepts/reasons-about-cost

Page: https://schubergphilis.github.io/ai-training/competencies/concepts/reasons-about-cost/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

*Concepts* · competency `concepts/reasons-about-cost`

**Taught in:** the [Concepts](https://schubergphilis.github.io/ai-training/concepts/) course

**Draws on:** [Tokens, cost and effort](https://schubergphilis.github.io/ai-training/topics/concepts/tokens-and-cost/), [What an agent is](https://schubergphilis.github.io/ai-training/topics/concepts/what-is-an-agent/)

## Learning objectives

### Reads the token counts of a request and says where its cost comes from (base)

| Claim | Why | Example |
| ----- | --- | ------- |
| The learner tells the input tokens from the output tokens in a usage report and knows that an output token costs more. | The two kinds are priced differently, so the same total can mean very different bills (Pricing, Claude Platform documentation). | A summary job reports 40,000 input and 500 output tokens, and the learner sees that the long document, and not the short summary, is most of the cost. |
| The learner finds the part of the input that was read from the cache. | Input read from the cache is billed at a small fraction of the normal price, so the input count alone overstates the cost (Prompt caching, Claude Platform documentation). | A report shows 30,000 tokens read from the cache and 800 new input tokens, and the learner explains that most of the request was the same as the one before. |
| The learner looks at the counts before guessing why a task was expensive. | The counts show which part of the request grew, and a guess usually blames the wrong part. | The learner expected the long answers to be the problem, and the counts show the cost is a large document attached to every message. |

Served by: [Reading the token counts](https://schubergphilis.github.io/ai-training/concepts/reading-token-counts/)

### Explains why a session costs more per request as it grows (base)

| Claim | Why | Example |
| ----- | --- | ------- |
| The learner explains that the tool sends the whole conversation with every request, so each request carries all the text before it. | The model is stateless, so a long history is paid for again on every request (Manage costs effectively, Claude Code documentation). | Asked why a one-line question late in a long chat used so many tokens, the learner says the question was sent together with an hour of history. |
| The learner starts a new conversation for an unrelated task. | The old history adds to the cost of every new request and does not help with the new task. | After finishing the report, the learner opens a new chat for the unrelated email instead of continuing in the long one. |
| The learner counts the requests an agent sends, which is more than the prompts they typed. | An agent sends a new request after each tool result, and each of those carries the history (How the agent loop works, Claude Code documentation). | The learner sees one instruction lead to a dozen tool calls and expects about a dozen requests in the usage report. |

Served by: [Reading the token counts](https://schubergphilis.github.io/ai-training/concepts/reading-token-counts/)

### Chooses an effort level for a kind of work and keeps it fixed (base)

| Claim | Why | Example |
| ----- | --- | ------- |
| The learner picks a low effort level for simple, well-defined work and a higher one for hard or open problems. | Higher effort spends more tokens on thinking and on tool use, and simple work gains nothing from that (Effort, Claude Platform documentation). | Reformatting a list of addresses runs at low effort, and comparing the terms of two contracts runs at high effort. |
| The learner tries a level on a few real examples of the work before settling on it. | The right level depends on the task, and the vendor's own advice is to test on your own work. | The learner runs three typical requests at medium and at high effort, sees no difference in quality, and keeps medium. |
| The learner keeps the level fixed within one conversation and changes it only between kinds of work. | Changing the effort level between requests can invalidate the prompt cache, so the next request writes the whole history to the cache again, at more than the normal input price (Effort, Claude Platform documentation) (Pricing, Claude Platform documentation). | The learner sets the level when starting the review session and leaves it there, and uses another level in a separate drafting session. |

Served by: [Caching and effort](https://schubergphilis.github.io/ai-training/concepts/caching-and-effort/)

### Explains what prompt caching saves and what breaks it (expert)

| Claim | Why | Example |
| ----- | --- | ------- |
| The learner explains that a request reuses the cache only when it starts with exactly the same tokens as a recent request. | The cache stores the opening part of a request, and any difference ends the match at that point (Prompt caching, Claude Platform documentation). | The learner explains why two requests with the same long document but a different first instruction both write the document to the cache instead of reading it. |
| The learner puts content that stays the same at the start of a request and content that changes at the end. | A change invalidates the cache from that point on, so changing text near the start wastes everything after it. | The learner puts the shared reference document before the question, so the next question reuses the document from the cache. |
| The learner names what breaks the cache: changed instructions or tools, a changed setting such as the effort level, and a pause longer than the cache lifetime. | Each of these makes the next request pay again for everything from the change on (Manage costs effectively, Claude Code documentation). | After a long lunch break, the first message in the old session costs much more than the one before the break, and the learner knows why. |

Served by: [Caching and effort](https://schubergphilis.github.io/ai-training/concepts/caching-and-effort/)

## References

- Pricing, Claude Platform documentation, https://platform.claude.com/docs/en/about-claude/pricing
- Prompt caching, Claude Platform documentation, https://platform.claude.com/docs/en/build-with-claude/prompt-caching
- Manage costs effectively, Claude Code documentation, https://code.claude.com/docs/en/costs
- How the agent loop works, Claude Code documentation, https://code.claude.com/docs/en/agent-sdk/agent-loop
- Effort, Claude Platform documentation, https://platform.claude.com/docs/en/build-with-claude/effort