# Tokens, cost and effort

> Every request to a model is counted and billed in tokens. This topic covers the counts a request reports, why each request in a long session costs more than the one before, how prompt caching makes an unchanged start of a request cheap, and how an effort setting trades thoroughness against tokens.

Page: https://schubergphilis.github.io/ai-training/topics/concepts/tokens-and-cost/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

*Concepts* · topic `concepts/tokens-and-cost`

Every request to a model is counted and billed in tokens. This topic covers the counts a request reports, why each request in a long session costs more than the one before, how prompt caching makes an unchanged start of a request cheap, and how an effort setting trades thoroughness against tokens.

## Concepts

- **Input and output tokens**: The two counts every response reports. Input tokens are everything the request sent: instructions, tool descriptions, the history and the new message. Output tokens are what the model wrote, including tool requests and any thinking, and each one costs several times as much as an input token. With prompt caching, the input count is split further into tokens written to the cache, tokens read from it and the rest. Reading these counts shows where the cost of a request comes from. [glossary](https://schubergphilis.github.io/ai-training/glossary/#input-and-output-tokens)
- **Growing session cost**: Because the model is stateless, each request in a session sends the whole conversation again, and an agent sends a new request after every batch of tool results. So each request carries everything before it, and a question late in a long session costs more than the same question in a new one. Starting fresh for an unrelated task resets the history, and compacting shrinks it at the price of one large summary request. [glossary](https://schubergphilis.github.io/ai-training/glossary/#growing-session-cost)
- **Prompt caching**: Reusing the processing of the opening part of a request when a later request starts with exactly the same tokens. The reused part is billed at a small fraction of the normal input price. The opening part has a fixed order: tool descriptions, then the system prompt, then the messages. After a change, everything from that point on is cached again, at more than the normal input price. An unused cache expires after five minutes, or after one hour at a higher write price. [glossary](https://schubergphilis.github.io/ai-training/glossary/#prompt-caching)
- **Effort level**: A setting that tells the model how many tokens to spend on a response, from low to max. Higher effort gives more thinking and more thorough tool use, and costs more tokens and time. Lower effort is faster and cheaper and suits simple work. The setting guides the model but does not set a hard limit, and on at least one model it does not reliably change the length of the answer. Changing it in one conversation can invalidate the prompt cache. [glossary](https://schubergphilis.github.io/ai-training/glossary/#effort-level)

## Links

- **Builds on:** [How language models work](https://schubergphilis.github.io/ai-training/topics/concepts/how-models-work/)
- **Leads to:** nothing yet
- **Related:** [Capabilities and limits](https://schubergphilis.github.io/ai-training/topics/concepts/limits/), [Choosing models and tools](https://schubergphilis.github.io/ai-training/topics/using-agents/choosing-tools/)
- **Competencies drawing on it:** [Reasons about what a model request costs and why](https://schubergphilis.github.io/ai-training/competencies/concepts/reasons-about-cost/)

## Lessons

- [Caching and effort](https://schubergphilis.github.io/ai-training/concepts/caching-and-effort/) (explanation)
- [Reading the token counts](https://schubergphilis.github.io/ai-training/concepts/reading-token-counts/) (tutorial)

## Sources

- `Claude docs prompt-caching` [Prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching), Claude Platform documentation (reference)
- `Claude Code costs` [Manage costs effectively](https://code.claude.com/docs/en/costs), Claude Code documentation (reference)
- `Claude docs effort` [Effort](https://platform.claude.com/docs/en/build-with-claude/effort), Claude Platform documentation (reference)
- `Claude docs messages` [Messages](https://platform.claude.com/docs/en/api/messages), Claude Platform documentation (reference)
- `Claude docs pricing` [Pricing](https://platform.claude.com/docs/en/about-claude/pricing), Claude Platform documentation (reference)
- `OpenAI Academy optimize-ai-application-performance-1k50h` [Optimize AI Application Performance](https://academy.openai.com/public/courses/optimize-ai-application-performance-1k50h), OpenAI Academy (course)