# Several tool calls in one turn

> Let the fake model ask for three weather lookups in one reply, run them together and return the results as one message, then watch a fourth call that needs the first result wait for its own round.

Page: https://schubergphilis.github.io/ai-training/building-agents/parallel-tool-calls/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

In this lesson the model asks for three things at once. The question
"Compare the weather in Amsterdam, Lisbon and Oslo" needs three lookups
that have nothing to do with each other, and a loop that takes one call per
turn spends a model round on each. We change the loop so a reply holds a
list of calls, and the loop runs the whole list before it returns every
result in one message. Then we add a call that needs one of those results
and see why it waits for the next round.

Every example runs against a fixture with a fake model, so you can predict
the output and check yourself. The complete program is
`site/examples/building-agents/parallel-tool-calls/agent.py` in the
repository. Its `run` function is the loop from
[When a tool fails](https://schubergphilis.github.io/ai-training/building-agents/tool-errors/) with one change: a
model reply holds a list under `calls`, each call has an `id`, and the tool
message holds a list of results that name the `id` they answer. The
`execute` parameter of `run` decides how the list runs, and the fake models
read the message list to see which results they already have. The fixture
prints one line per message, then the round count, the stop reason, and the
answer. A round is one call to the model.

## One call per turn

The first fake model asks for one city at a time. It reads the weather
results it has and asks for the first city that's missing. Once it has all
three, it answers. A real model behaves this way when the request sets
`disable_parallel_tool_use` to `true`, and the loop from
[When a tool fails](https://schubergphilis.github.io/ai-training/building-agents/tool-errors/) only knew how to handle
one call per reply anyway.

```python
def model_one_at_a_time(messages):
    known = weather_so_far(messages)
    missing = [city for city in CITIES if city not in known]
    if missing:
        return {"calls": [weather_call(len(known) + 1, missing[0])]}
    return {"answer": summary(known)}
```

`weather_call(1, "Amsterdam")` builds
`{"id": "call_1", "tool": "get_weather", "args": {"city": "Amsterdam"}}`.
The loop counts its rounds and runs each reply's calls in the order they
appear, with `in_order`:

```python
def call_tool(call: dict) -> dict:
    tool = TOOLS[call["tool"]]
    return {"id": call["id"], "content": tool["fn"](**call["args"])}

def in_order(calls: list) -> list:
    return [call_tool(call) for call in calls]

def run(question, model, execute=in_order, max_steps=5):
    messages = [{"role": "user", "content": question}]
    rounds = 0
    for _ in range(max_steps):
        rounds += 1
        reply = model(messages)
        if "answer" in reply:
            return outcome("end_turn", reply["answer"], rounds, messages)
        results = execute(reply["calls"])
        messages.append({"role": "assistant", "content": reply})
        messages.append({"role": "tool", "content": results})
    return outcome("max_steps", None, rounds, messages)
```

The `show` function prints the `user` line as the question in `repr`, the
same as in "When a tool fails", an assistant line as
`call_1 get_weather(city='Amsterdam')` and a tool line as
`call_1 '14°C, rain'`. If a message has two or more calls or results, the
line lists them all, separated by commas. The weather table has
`Amsterdam: 14°C, rain`, `Lisbon: 27°C, sun` and `Oslo: 6°C, snow`, and
`summary` joins them in that order with a period after each.

#### Checkpoint: Predict the rounds, one call per turn

What does this print?

```python
show(run(QUESTION, model=model_one_at_a_time))
```

The three lookups took four rounds. Each round is a full trip to the
model with the whole message list, and those trips cost most of the time
and the tokens. If each lookup took a second and each model round took
two, the run would take about eleven seconds, and eight of them are spent
waiting for the model.

## All three in one reply

The second fake model asks for every city in its first reply. The
same loop handles that, because `execute` already takes a list.
The list can run in a different way, though. `together` hands the calls to
a thread pool, and `map` returns the results in the order of the calls, whichever
one finishes first.

```python
from concurrent.futures import ThreadPoolExecutor

def model_all_at_once(messages):
    known = weather_so_far(messages)
    if not known:
        return {"calls": [weather_call(number, city) for number, city in enumerate(CITIES, 1)]}
    return {"answer": summary(known)}

def together(calls: list) -> list:
    with ThreadPoolExecutor() as pool:
        return list(pool.map(call_tool, calls))
```

The loop appends one assistant message with the three calls and one tool
message with the three results.

#### Checkpoint: Predict the rounds, three calls in one reply

What does this print?

```python
show(run(QUESTION, model=model_all_at_once, execute=together))
```

The same answer came back in two rounds instead of four. With the
timings from the last section the run takes about five seconds. The three lookups
overlap, so they cost one second together, and the model is asked twice.
The vendor page says independent reads, such as three files the model
wants to see, are usually safe to run together
(Parallel tool use, Claude Platform documentation). A model that asks for them in one reply
saves the round trips in between (Tools, giving agents hands: function calling, schema design, the N x M problem, Agent Engineer Course).

The Claude API works the same way (Parallel tool use, Claude Platform documentation). A
reply with `stop_reason` `tool_use` can hold several `tool_use` blocks,
each with its own `id`. Your loop answers with one `user` message that
holds one `tool_result` block per `tool_use` block, each naming its
`tool_use_id`, and any text in that message comes after the results
(Handle tool calls, Claude Platform documentation). Whether those calls ran one after
another or in a thread pool is invisible in that message, so `in_order`
and `together` both produce a valid reply. When a model stops asking for
several tools at once, the docs name one result per `user` message as the
most common cause: a history in that format teaches the model to stop
asking for several at once (Parallel tool use, Claude Platform documentation). To
turn the batching off, set `disable_parallel_tool_use: true` inside the
`tool_choice` object.
With the default `tool_choice` type `auto` the model then makes at most
one tool call per reply, and the run looks like the first section.

## A call that needs a result

The third fake model wants to add packing advice for Amsterdam. The
`suggest_clothing` tool takes a weather report as its argument, so the
model can't ask for it in the first reply: the argument is a result it
doesn't have yet. It asks for the three lookups, then for the advice, then
answers.

```python
def model_with_follow_up(messages):
    known = weather_so_far(messages)
    if not known:
        return {"calls": [weather_call(number, city) for number, city in enumerate(CITIES, 1)]}
    requests, results = results_by_call(messages)
    advice = [text for call_id, text in results.items() if requests[call_id]["tool"] == "suggest_clothing"]
    if not advice:
        call = {"id": "call_4", "tool": "suggest_clothing", "args": {"weather": known["Amsterdam"]}}
        return {"calls": [call]}
    return {"answer": f"{summary(known)} Pack {advice[0]} for Amsterdam."}
```

`suggest_clothing` returns `"a raincoat"` for a report that mentions rain.

#### Checkpoint: Predict the rounds with a dependent call

What does this print?

```python
show(run(QUESTION, model=model_with_follow_up, execute=together))
```

The four calls took three rounds. The batch saved two rounds on the lookups,
and the dependent call cost one round of its own, because a round is the
only place where a result can flow into an argument. The rule for what
goes in one reply is the data flow: calls whose arguments are all known
now can share a reply, and a call whose argument is another call's result
waits for the reply after it.

A model doesn't always get this right, and a dependent call can arrive in
the same batch as the call it needs. The docs give two cases. For a call
you choose not to run, return a `tool_result` with `is_error` set to
`true` and a short note. For a call that ran and failed because the call
it needed hadn't finished, return the error with `is_error` set to `true`,
and the docs say the model makes the call again in the next turn. They
also suggest a line in the system prompt that asks the
model to batch only calls that don't depend on each other
(Parallel tool use, Claude Platform documentation). The exercise below builds that case.

#### Pitfall: Returning results in the order they finished

The thread pool also offers `as_completed`, which yields each result as
its call finishes. Say the Amsterdam lookup is slow: the list comes back
as Lisbon, Oslo, Amsterdam. Pair that list with the calls by position and
the model learns that Amsterdam has 27°C and sun and that Oslo has rain.
Rule: every result names the `id` of the call it answers, and the model
reads the pairing from the ids, never from the position.

## Side effects under concurrency

A weather lookup is safe to run at any time and in any order, because it
doesn't change anything. Tools that write are different. Two appends to the same
file end up in either order when they run in two threads, and a write
followed by a read of the same record can return the old value if the
read runs first. The choice of `in_order` or `together` is a property of
the tools in the batch (Parallel tool use, Claude Platform documentation). A harness that
knows which tools only read can use both strategies in one round: the
read-only calls go to the thread pool, and the writes wait their turn. The
reply still holds one result per call, whichever way each call ran.

#### Checkpoint: Which pair can run together?

The model asked for two tool calls in one reply. Which pair is safe to
run at the same time?

- Fetch the weather for Amsterdam and fetch the weather for Lisbon.
- Create an order and cancel the order with the id the first call returns.
- Write a new value to a settings record and read that record back.
- Append a line to a log file and append a second line to the same file.

## Exercise

Copy `agent.py` from the repository. Change `model_with_follow_up` so its
first reply holds four calls, the three lookups and a `suggest_clothing`
call whose `weather` argument is the string `"call_1"`, the way a model
does when it guesses that it can refer to a result it doesn't have. Run
`show(run(QUESTION, model=model_with_follow_up, execute=together))` and
look at what `suggest_clothing` returns for that string. Then change
`call_tool` so a call whose argument is another call's id isn't run and
returns `{"id": ..., "content": "not run: depends on call_1", "is_error": True}`
instead, and make the fake model treat that result as missing advice so
it asks again with the real report. Run again and compare both message
lists and round counts with the lesson's follow-up run. Doing this once
shows you what the harness can and can't do about a dependent call: it
can refuse to run it, and only the next round can supply the argument.

A good result: the first run answers with advice for the string `call_1`
in two rounds, which is wrong and fast. The second run shows the error
result for `call_4` in the first tool message, then a second reply with
the real `weather` argument, and three rounds, the same count as the
lesson's follow-up run. Your fake model asks again by construction. A
real model may, and for a call you didn't run the docs promise no retry, so
the harness can't count on it. What would the harness need to know to tell a dependent
call from an independent one without a rule about ids in arguments?

Stretch: Add a third strategy to the fixture, next to in_order and together: run every get_weather call in the thread pool and every other call one after another, in the same round. Check that the follow-up run's output stays the same.

## Recap

1. A model reply can hold several tool calls at once. The loop runs the
   list and returns one result per call, all in the next message, each
   under the id of the call it answers (Parallel tool use, Claude Platform documentation).
2. Three independent lookups cost two rounds in a batch and four one at
   a time. The batch saves model round trips, and running the
   calls in a thread pool saves the wait for the tools too (Tools, giving agents hands: function calling, schema design, the N x M problem, Agent Engineer Course).
3. A call whose argument is another call's result waits for the next
   round. Data flow decides what can share a reply.
4. Running a batch at the same time is safe for calls that only read. For
   calls that write, run them in order, and keep the ids on every result
   either way.

## References

- Parallel tool use, Claude Platform documentation, https://platform.claude.com/docs/en/agents-and-tools/tool-use/parallel-tool-use
- Tools, giving agents hands: function calling, schema design, the N x M problem, Agent Engineer Course, https://lsimons.github.io/agent-engineer-course/03-tools-giving-agents-hands/
- Handle tool calls, Claude Platform documentation, https://platform.claude.com/docs/en/agents-and-tools/tool-use/handle-tool-calls