# Writing a tool schema the model uses correctly

> Give a fake model two overlapping tools, watch it pick the wrong one, and fix that with a description that says when to use the tool and an enum the model cannot guess around.

Page: https://schubergphilis.github.io/ai-training/building-agents/tool-schemas/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

In [Building your first agent](https://schubergphilis.github.io/ai-training/building-agents/agent-loop/) the model had
one tool, so its only choice was between calling it and answering. In this
lesson the model gets two tools that overlap. We watch it call the wrong one
and fix that without touching the functions. Everything we change is in
the schema: the description the model reads and the parameters it fills in.

Every example runs against a fixture: a fake model that behaves
deterministically, so you can predict what happens and check yourself. The
complete program is `site/examples/building-agents/tool-schemas/tools.py`
in the repository, and each step below is `python3 tools.py <step>`.

## What the model reads

A tool definition in the Claude API has a `name`, a `description`, and an
`input_schema`, which is a JSON Schema object that names each parameter
and its type (Define tools, Claude Platform documentation). The API requires the name and
the schema and accepts a tool without a description (Messages, Claude Platform documentation),
but then the model has only the name and the schema to go on. The API
builds a system prompt from the tool definitions (Define tools, Claude Platform documentation),
and your code sends them again with every request, because the model
doesn't keep state between calls (Building with the Claude API, Claude Academy).
That text is all the model has when it decides which tool to call and what
to put in the arguments. The function body never reaches it.

A first draft of two customer tools often looks like this. The functions
are stubs, because this lesson is about the text around them.

```python
VAGUE_TOOLS = [
    {"name": "get_customer", "description": "Get customer.", "fn": get_customer},
    {
        "name": "search_customers",
        "description": "Search customers. Matches name, email, city or account id.",
        "fn": search_customers,
    },
]
```

The fixture's fake model chooses a tool the way a hurried reader would. It
lowercases the request and each description, drops ten filler words such
as `the` and `by`, strips a plural `s`, and counts the words a request
shares with each description. The model picks the tool with the most
shared words, and a tie goes to the first tool in the list. A tool name
such as `search_customers` counts as one word.

```python
FILLER = {"a", "an", "the", "by", "for", "or", "and", "to", "of", "with"}


def words(text):
    return {w.rstrip("s") for w in re.findall(r"[a-z0-9_]+", text.lower())} - FILLER


def choose_tool(request, tools):
    scores = [(len(words(request) & words(tool["description"])), tool["name"]) for tool in tools]
    best = max(score for score, _ in scores)
    return next(name for score, name in scores if score == best)
```

A real model weighs meaning, and it reads a description the way it reads
any other prompt. The counting model is a caricature, but it fails on the
same descriptions a real model fails on, and you can work out its choice
by hand.

#### Checkpoint: Which tool gets called?

Which tool name does this print?

```python
print(choose_tool("Look up the customer with account id 4711", VAGUE_TOOLS))
```

The request is a lookup by account id, which is the one job `get_customer`
exists for. Its description shares one word with the request. The other
description happens to mention account ids, so it shares three, and the
wrong tool runs a text search over a request that named an exact id. The
function was never at fault. The model did what the text told it to.

## Say when to use it, and when not to

Rewrite the description of `get_customer`. Leave `search_customers` as it
is for now, so you can see what one description does on its own.

```python
CLEAR_TOOLS = [
    {
        "name": "get_customer",
        "description": (
            "Look up one customer by exact account id, an integer such as 8842. "
            "Returns the full record for that id. "
            "Use search_customers instead when you do not have the account id."
        ),
        "fn": get_customer,
    },
    VAGUE_TOOLS[1],
]
```

Each sentence gives the model something it needs. The first says what the
tool does and what the argument looks like, with an example value. The
second says what comes back. The third says when to use the other tool.
Anthropic's guidance for real tool definitions asks the description to
cover the following, in at least three or four sentences
(Define tools, Claude Platform documentation):

- what the tool does
- when it should be used, and when it shouldn't
- what each parameter means and how it changes the result
- any caveats or limits

The same page puts a detailed description first among its tips, and says
no other part of a tool definition matters as much for how well the model
uses the tool (Define tools, Claude Platform documentation).

Look at how the third sentence is written. It names the other tool and the
input you lack, and it doesn't repeat that tool's own trigger words. The
fake model can't read the word `not`. Every word in a description makes
the tool a closer match for requests that contain that word. A real model reads `not` most of
the time, but a description that lists what the tool isn't for still adds
those words to the text the model weighs.

#### Checkpoint: Predict again

The same request against the rewritten tools. Which tool name does this
print?

```python
print(choose_tool("Look up the customer with account id 4711", CLEAR_TOOLS))
```

## A parameter the model has to guess

Tool choice is half of a call. The other half is the arguments, and the
model gets those from the parameter schema. Here is a balance tool whose
data is keyed by lowercase currency codes, with a schema that says only
that `currency` is a string.

```python
BALANCES = {(4711, "eur"): "250.00", (4711, "usd"): "271.50", (4711, "gbp"): "214.20"}


def get_balance(account_id, currency):
    amount = BALANCES.get((account_id, currency))
    if amount is None:
        return f"unknown currency: {currency}"
    return f"{account_id}: {amount} {currency}"


FREE_TEXT_SCHEMA = {
    "type": "object",
    "properties": {
        "account_id": {"type": "integer"},
        "currency": {"type": "string"},
    },
    "required": ["account_id", "currency"],
}
```

To fill `currency`, the fake model finds the currency word in the request
and then looks at that parameter's schema. Without a list of allowed
values it writes the first three letters as a code in capitals, because
most text it has seen writes codes that way. A real model has the same
habit, and a free-text parameter gives it no reason to write anything
else. `call_balance` plays the loop's part: it fills both arguments from
the request and runs the tool.

```python
CURRENCY_WORDS = {"euros", "euro", "dollars", "dollar", "pounds", "pound"}


def fill_currency(request, schema):
    mentioned = next(
        w.rstrip(".?") for w in request.lower().split() if w.rstrip(".?") in CURRENCY_WORDS
    )
    if "enum" in schema:
        return next(code for code in schema["enum"] if mentioned.startswith(code))
    return mentioned[:3].upper()


def call_balance(request, schema):
    digits = re.search(r"\d+", request)
    if digits is None:
        return "no account id in the request"
    account_id = int(digits.group())
    currency = fill_currency(request, schema["properties"]["currency"])
    return get_balance(account_id, currency)
```

#### Checkpoint: Predict the currency

What does this print?

```python
print(call_balance("What is the balance of account 4711 in euros?", FREE_TEXT_SCHEMA))
```

The fix is in the schema. JSON Schema has an `enum` keyword that restricts
a value to a fixed list, and a `description` on each parameter tells the
model what the value means. Anthropic's own example does this for a
temperature unit, and with `strict: true` on the tool definition the API
constrains the model's output so the arguments always match the schema.
Without it the model can still return the wrong type, such as the string
`"2"` for an integer, or leave out a required field
(Define tools, Claude Platform documentation) (Strict tool use, Claude Platform documentation).

```python
ENUM_SCHEMA = {
    "type": "object",
    "properties": {
        "account_id": {
            "type": "integer",
            "description": "The customer's account id, such as 4711.",
        },
        "currency": {
            "type": "string",
            "enum": ["eur", "usd", "gbp"],
            "description": "Lowercase currency code. The balance is converted into this currency.",
        },
    },
    "required": ["account_id", "currency"],
}
```

The fake model reads the enum and picks the allowed value the request
mentions. `get_balance` is unchanged.

#### Checkpoint: Predict with the enum

Same request, the enum schema. What does this print?

```python
print(call_balance("What is the balance of account 4711 in euros?", ENUM_SCHEMA))
```

#### Checkpoint: Fix the schema

This is the `input_schema` of a `charge_customer` tool. The tool stores
amounts in whole cents and accepts the codes `eur`, `usd` and `gbp`, and
account ids are integers. Rewrite the schema so the model can't invent a
value for any of the three parameters.

```text
{
"type": "object",
"properties": {
  "account_id": {"type": "string"},
  "amount": {"type": "number"},
  "currency": {"type": "string"}
},
"required": ["account_id", "amount", "currency"]
}
```

## The description is a prompt

The rule behind both fixes is the same. The schema is the only prompt the
model gets about a tool, so tool schema design is prompt writing. The
description says what the tool does and what comes back, and it says when
to use this tool and when to use another one. Each parameter gets a type
and a description, and an enum where the set is small. When a tool gets called for the wrong job or with
a value it can't use, read the schema before you suspect the model
(Tools, giving agents hands: function calling, schema design, the N x M problem, Agent Engineer Course).

The fixture keeps six requests and the tool each one should reach, and
counts how many the fake model gets wrong under each set of descriptions.
Run it.

#### Example: Count the wrong calls

Run this, and compare what you see with the output below.

```sh
python3 tools.py count
```

Output:

```text
wrong calls before: 3 of 6
wrong calls after: 1 of 6
still wrong: Show the email and current balance of account id 4711
```

The rewrite fixed the plain lookups and left one request wrong. That
request asks for two fields of one customer's record by account id. The
rewritten description says "the full record" and doesn't name a field.
The request still shares more words with the search tool. The count is the
evidence you want when a description is under discussion. The model's
choices are cheap to measure, and the fix is a sentence.

#### Pitfall: Describing the implementation instead of the use

A first draft of a lookup tool often reads "Runs `SELECT * FROM customers
WHERE id = ?` against the billing replica." Every word in it is true, and
almost none of them appears in a request. Against the fixture's six
requests it shares at most two words with any of them. All three lookups
still go to the search tool, the same count that "Get customer." gets.
Rule: write the description for the caller, from the request side. Say
what to ask this tool for, in the words a request uses. Say what comes
back.

## Exercise

Copy `tools.py` from the repository. Rewrite the `get_customer`
description in `CLEAR_TOOLS`, and leave `search_customers` as it is, until
`python3 tools.py count` reports `wrong calls after: 0 of 6`. Every change
you make is a sentence, so keep the before and after descriptions to show
what moved the count.

A good result: a description that also says which fields the record
contains, such as the customer's name, email address and current balance,
so that a request for one of those fields by account id shares more words
with the lookup tool than with the search tool. Which single word did the
work, and would a real model have needed it?

Stretch: Send the same six requests through a real model API with the CLEAR_TOOLS descriptions as its tool definitions, and compare its wrong-call count with the fixture's.

## Recap

1. The schema is the only prompt the model gets about a tool. A
   description says what the tool does, when to use it, when to use
   another tool, and what comes back (Define tools, Claude Platform documentation).
2. A parameter gets a type and a description, and an enum where the set
   is small, so the model has nothing to guess.
3. Count the wrong calls before and after a change, because when the
   description doesn't say when to use a tool, the model uses it wrongly or
   not at all (Tools, giving agents hands: function calling, schema design, the N x M problem, Agent Engineer Course).

## References

- Define tools, Claude Platform documentation, https://platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools
- Messages, Claude Platform documentation, https://platform.claude.com/docs/en/api/messages
- Building with the Claude API, Claude Academy, https://academy.claude.com/courses/building-with-the-claude-api
- Strict tool use, Claude Platform documentation, https://platform.claude.com/docs/en/agents-and-tools/tool-use/strict-tool-use
- Tools, giving agents hands: function calling, schema design, the N x M problem, Agent Engineer Course, https://lsimons.github.io/agent-engineer-course/03-tools-giving-agents-hands/