# Approving a risky tool call

> Make the loop stop before a refund tool runs, show a person the call and its exact arguments, and send a refusal back to the model as the tool's result.

Page: https://schubergphilis.github.io/ai-training/building-agents/approving-a-tool-call/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

In this lesson we give the agent a tool that changes data and make the loop
stop before it runs. A person sees the call with its exact arguments and
says yes or no. When the answer is no, the loop sends the refusal to the
model as the tool's result. The model can then tell the user what happened,
and the run ends the normal way.

[Agent risk](https://schubergphilis.github.io/ai-training/safety/agent-risk/#human-in-the-loop) asked where a person
must approve. This lesson builds that approval step into the loop.

Every example runs against a fixture with a fake model, so you can predict
the output and check yourself. The programs are in
`site/examples/building-agents/approving-a-tool-call/` in the repository.
`start.py` has no approval step, and `agent.py` adds it. Both use the loop
from [When a tool fails](https://schubergphilis.github.io/ai-training/building-agents/tool-errors/), with its
`is_error` flag and its `max_errors` policy, and two new tools for a
support agent.

## A refund nobody checked

The agent has two tools. `lookup_order` reads an order, and `issue_refund`
sends money back to the customer. In the fixture the refund adds a line to
a list called `REFUNDS`, so you can see whether it ran.

```python
ORDERS = {"A17": {"item": "kettle", "paid": "40.00 EUR"}}
REFUNDS = []

def lookup_order(order_id: str) -> dict:
    order = ORDERS.get(order_id)
    if order is None:
        return {"ok": False, "error": f"no order {order_id}"}
    return {"ok": True, "item": order["item"], "paid": order["paid"]}

def issue_refund(order_id: str, amount: str) -> dict:
    REFUNDS.append((order_id, amount))
    return {"ok": True, "refunded": amount}
```

The fake model looks the order up first. Then it asks for a refund of the
amount that the customer's message names, without comparing it with what
the customer paid. A real model can make the same mistake, or a customer
can type a wrong number. The question in this run asks for 400.00 euros (EUR)
for a kettle priced at 40.00 EUR. The fixture prints the message list as in the
last lesson, then the stop reason, the answer and the contents of
`REFUNDS`.

#### Example: The refund without a check

Run the program without an approval step.

```sh
python3 start.py
```

Output:

```text
user: 'Order A17 arrived broken, please refund 400.00 EUR.'
assistant: {'tool': 'lookup_order', 'args': {'order_id': 'A17'}}
tool: {'ok': True, 'item': 'kettle', 'paid': '40.00 EUR'}
assistant: {'tool': 'issue_refund', 'args': {'order_id': 'A17', 'amount': '400.00 EUR'}}
tool: {'ok': True, 'refunded': '400.00 EUR'}
stop: end_turn
answer: Refunded 400.00 EUR for order A17.
refunds: [('A17', '400.00 EUR')]
```

The order lookup shows that the customer paid 40.00 EUR, and the model
refunded 400.00 EUR anyway. The loop did what it was built to do: it ran
every call the model asked for. A wrong lookup doesn't change anything, but this
wrong refund sends the customer more money than they paid, and that money
is hard to get back.

## Pause before the call that changes data

Tool approval puts a person between the model's request and the tool. The
harness decides which tools need it. Each tool entry gets a flag for that.
`lookup_order` doesn't change anything and runs freely. `issue_refund` moves money
and waits.

```python
TOOLS = {
    "lookup_order": {"fn": lookup_order, "description": "...", "needs_approval": False},
    "issue_refund": {"fn": issue_refund, "description": "...", "needs_approval": True},
}
```

The approval step shows the person the tool name and every argument
exactly as the model sent it, one per line. Then it asks. Any answer other
than `y` is a refusal, and a refusal asks for a reason. When the input ends
before an answer, the step refuses with the reason `no answer`.

```python
def ask_person(name, args, read=input) -> dict:
    print(f"approve? {name}")
    for key, value in args.items():
        print(f"  {key} = {value!r}")
    try:
        if read("allow? [y/n] ").strip() == "y":
            return {"allow": True}
        return {"allow": False, "reason": read("reason: ").strip()}
    except EOFError:
        return {"allow": False, "reason": "no answer"}
```

Before a flagged tool, the loop asks, and the tool runs only after a yes.
After a no, the loop never calls the tool and builds an error result from
the reason, in the same `ok: False` form that the last lesson used for
a tool that failed.

```python
tool = tools[reply["tool"]]
decision = {"allow": True}
if tool["needs_approval"]:
    decision = approve(reply["tool"], reply["args"])
if decision["allow"]:
    result = tool["fn"](**reply["args"])
else:
    result = {"ok": False, "error": f"Refused by the reviewer: {decision['reason']}"}
```

`read` is `input` when a person is at the keyboard. The fixture steps pass
a stand-in that gives a fixed answer and prints it after the prompt, the
way a terminal shows what you typed. So continuous integration (CI) can
run both answers without a person. The approval lines print while the loop
runs, before the fixture prints the message list. Here a person approves a
refund of the right amount.

#### Example: An approved refund

Run the `approved` step with `python3 agent.py approved`. It asks for the
right amount and answers `y`.

```python
def reviewer(name, args):
    return ask_person(name, args, read=typed("y"))

show(run("Order A17 arrived broken, please refund 40.00 EUR.", approve=reviewer))
```

Output:

```text
approve? issue_refund
  order_id = 'A17'
  amount = '40.00 EUR'
allow? [y/n] y
user: 'Order A17 arrived broken, please refund 40.00 EUR.'
assistant: {'tool': 'lookup_order', 'args': {'order_id': 'A17'}}
tool: {'ok': True, 'item': 'kettle', 'paid': '40.00 EUR'}
assistant: {'tool': 'issue_refund', 'args': {'order_id': 'A17', 'amount': '40.00 EUR'}}
tool: {'ok': True, 'refunded': '40.00 EUR'}
stop: end_turn
answer: Refunded 40.00 EUR for order A17.
refunds: [('A17', '40.00 EUR')]
```

The approval prompt appeared once, for the refund. The lookup ran without
one, because its flag is `False`.

## A refusal is a tool result

Now the customer asks for 400.00 EUR again, and the reviewer checks the
amount against the order. The fake model reads a result with `ok` set to
`False` the way it did in the last lesson. Its answer is `I did not
refund order A17.` followed by a space, the error text and a full stop.

#### Checkpoint: Predict the refused run

The `refused` step (`python3 agent.py refused`) answers `n` and gives a
reason. What does it print?

```python
def reviewer(name, args):
    return ask_person(name, args, read=typed("n", "the customer paid 40.00 EUR"))

show(run("Order A17 arrived broken, please refund 400.00 EUR.", approve=reviewer))
```

`REFUNDS` is empty, so no money moved. The run still ended with `end_turn`,
and the user heard why. A refusal that raised an exception would have ended
the run with a stack trace, and a loop that stopped without a word would
leave the user waiting. As a tool result, the refusal is one more fact the
model can act on. A real model reading `the customer paid 40.00 EUR` could
ask for the smaller refund on its next turn, and that call would come back
to the reviewer too. A refusal is an error result, so it counts toward
`max_errors`. If the reviewer refuses that second call as well, the run
stops with `too_many_errors` after two refusals in a row. That is the
policy from the last lesson doing its job: a model that keeps asking for
something a person keeps refusing doesn't get to ask forever.

Write the reason for the model. `no` tells it only that
something was wrong. `the order is over 90 days old, offer store credit`
tells it what to do instead.

## Which tools wait for a person

An approval prompt costs the reviewer's attention every time it appears.
If every lookup asks, people learn to press `y` without reading, and then
the prompt before the refund protects nothing. So sort the tools by what a
wrong call can do. A tool that only reads, or that writes to a draft the
agent keeps to itself, runs freely. A tool whose call is hard to undo, or
that reaches a customer, a shared system or money, waits for a person.
These are the first two of the questions that
[Agent risk](https://schubergphilis.github.io/ai-training/safety/agent-risk/#human-in-the-loop) used to choose where a
person approves.

#### Checkpoint: Which tools wait?

Buckets:

- Runs without asking
- Waits for a person

Items:

- Search the help articles
- Read an order
- Save a draft reply in the agent's notes
- Send the reply to the customer
- Issue a refund
- Close the customer's account

## What the person sees

An approval is only as good as what the reviewer can check. A prompt that
says `approve? issue_refund` asks the person to trust the model, because
the amount isn't on the screen. `ask_person` prints every argument as the
model sent it, which is exactly what will run. A longer argument, such as
the body of an email, belongs on the screen in full for the same reason.

#### Checkpoint: What goes on the approval prompt?

The refund tool waits for approval. What should the prompt show the
reviewer?

- The tool name with an allow button, so the person can answer fast.
- A one-line summary the model writes of what it is about to do.
- The tool name and every argument as the model sent it.
- The model's reasons for the refund, quoted from its reply.

## The same step in the Agent SDK

The Claude Agent software development kit (SDK) has this step built in.
You pass a `canUseTool` callback (`can_use_tool` in Python), and the SDK
calls it with the tool's name and the input the model sent. The callback
returns allow or deny. A deny carries a message, which goes to the model so
it can plan its next step. An allow can also carry a changed input. The
tool then runs with your version, and nothing tells the model about the
edit (Handle approvals and user input, Claude Code documentation).

The callback is the last step of the SDK's checks. Before it, hooks run,
then deny rules, ask rules, the permission mode and allow rules, and the
callback hears only about a call that none of those settled
(Configure permissions (Agent SDK), Claude Code documentation). So a check that you put only in the
callback never sees a call that an allow rule or a mode such as
`bypassPermissions` already approved. A check that must see every call
belongs in a `PreToolUse` hook, because hooks run first, before the rules
and the mode (Configure permissions (Agent SDK), Claude Code documentation). Your `needs_approval`
flag is the simple form of the same idea: the harness settles most calls by
rule, and only the rest wait for a person.

#### Pitfall: Asking in the prompt

A tempting shortcut is a line in the system prompt: "Always ask the user
before you issue a refund." On some runs the model asks in its reply. On
others it calls the tool straight away, and nothing in the loop stops it.
A question in the model's reply isn't a pause in the loop. Rule: the gate
is code in the harness that runs before the tool, so it holds whatever
the model decides.

## Exercise

Copy `start.py` from the repository. Add an approval step to its loop for
`issue_refund`, the one tool that changes data. Show the person the tool
name and its arguments, and when the person refuses, send the reason back
to the model as the tool's result. Then make the prompt easier to check:
print what the customer paid for the order next to the amount the model
asks for. Run your copy with `python3 start.py` and answer at the keyboard,
once with `y` and once with `n` and a reason of your own. Fifteen minutes
is enough. After this you can put a gate in any loop you build, so a
wrong call stops at a person instead of reaching a customer.

A good result: the lookup never asks. The refund prompt shows the order,
the amount asked for and the amount paid. After `y` the last line holds
the refund, and after `n` it is `refunds: []` and the answer holds your
reason. Compare your loop with `agent.py`, or run `python3 agent.py ask`
to try the lesson's version. Which other fact would you want on the screen
before you approve a refund, and where would the loop get it?

Stretch: Add a third tool, cancel_order, decide whether it needs approval, and add a branch to the fake model that calls it. Run it once and say why you set the flag the way you did.

## Recap

1. The harness marks the tools whose wrong call is hard to undo or reaches
   other people, and the loop asks a person before it runs one of those.
   Reads and private drafts run without asking.
2. The person sees the tool name and every argument as the model sent it,
   because that is what will run.
3. A refusal skips the tool and goes back to the model as the tool's
   result, with a reason the model can act on, so the run ends the normal
   way. It counts as an error, so refusals in a row hit `max_errors`.
4. The Agent SDK does the same with a `canUseTool` callback that returns
   allow or deny with a message (Handle approvals and user input, Claude Code documentation), and it
   calls the callback only for calls that its hooks, rules and mode
   didn't settle (Configure permissions (Agent SDK), Claude Code documentation).

#### Habit

The next time an agent asks you to approve an action, read the arguments
before you say yes, and refuse with a reason when one of them is wrong.

## References

- Handle approvals and user input, Claude Code documentation, https://code.claude.com/docs/en/agent-sdk/user-input
- Configure permissions (Agent SDK), Claude Code documentation, https://code.claude.com/docs/en/agent-sdk/permissions