# Orchestration in code or by a model

> Run three fake agents as a pipeline in code and then under a manager model that calls two of them as tools, name the sequential, hierarchical and collaborative arrangements, and stop the run with a report when a fault hits a boundary.

Page: https://schubergphilis.github.io/ai-training/building-agents/orchestrating-agents/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

In this lesson one customer message goes through three fake agents in two
arrangements. First a program runs the agents in a fixed order and passes
each output to the next. Then a fake manager model decides which agent
runs when, and it calls the other agents as tools. After that we name the
arrangements, and we break the manager version at each boundary to see
what reaches the customer.

The customer is from the small shop in
[Writing the plan before taking the steps](https://schubergphilis.github.io/ai-training/building-agents/planning/),
and the message is about the kettle from order 1042. The agents and the
manager are fake models written by hand. The runs show how each
arrangement passes work along, and they don't show how well a real model
does it. The complete program is
`site/examples/building-agents/orchestrating-agents/agent.py` in the
repository. Copy the `orchestrating-agents` folder, and each step below is
`python3 agent.py <step>`.

## A pipeline written in code

The message is "My kettle from order 1042 stopped working. Please send
the replacement to my office address this time." The agents are
`lookup`, `writer` and `checker`. `lookup` reads the message, looks up
the order and writes a brief of one line. `writer` writes the reply from
the brief. `checker` reads the reply and approves it or rejects it. Here
are the first two, each a fake model call with text in and text out.

```python
def lookup(message: str) -> str:
    """Reads the customer's message, looks up the order and writes a one-line brief."""
    order_id = order_id_in(message)
    order = ORDERS[order_id]
    wish = "refund" if "money back" in message else "replacement"
    days = order["days_since_delivery"]
    return f"order {order_id}, {order['item']}, delivered {days} days ago, wants a {wish}"


def writer(brief: str) -> str:
    """Writes the reply to the customer from the brief."""
    parts = brief.split(", ")
    if len(parts) == 4 and parts[0].startswith("order "):
        order, item, wish = parts[0], parts[1], parts[3].replace("wants a ", "")
        return (
            f"Dear customer, we are sorry that your {item} broke. "
            f"A {wish} for {order} is on its way."
        )
    return "Dear customer, we are sorry about the problem. We will get back to you soon."
```

The order data says the kettle was delivered 12 days ago. The program
decides who runs when, and it always runs `lookup`, then `writer`, then
`checker`.

```python
def pipeline(message: str) -> list:
    messages = [("customer", "lookup", message)]
    brief = lookup(message)
    messages.append(("lookup", "writer", brief))
    draft = writer(brief)
    messages.append(("writer", "checker", draft))
    verdict = checker(draft)
    messages.append(("checker", "program", verdict))
    if verdict == "approved":
        messages.append(("program", "customer", draft))
    return messages
```

This is orchestration in code, which the Agent Engineer Course calls
deterministic. The flow is written down in advance, so it runs the same
way every time and you can step through it like other code. Deciding
what comes next costs no model call (Orchestrators: code- versus model-driven, patterns, anti-patterns, Agent Engineer Course). The messages from one
agent to the next are the handoffs. The `handoffs` step prints those
two, each as `sender -> receiver: 'text'`.

#### Checkpoint: Predict the handoffs

What are the two handoff messages? Predict the output, then run it.

```sh
python3 agent.py handoffs
```

The brief has no place for the office address, so the address is lost at
the first boundary. The checker approves the reply, because it sees only
the draft, and the draft reads well. Here is the whole run.

#### Example: The whole pipeline

Run this, and compare what you see with the output below.

```sh
python3 agent.py pipeline
```

Output:

```text
customer -> lookup: 'My kettle from order 1042 stopped working. Please send the replacement to my office address this time.'
lookup -> writer: 'order 1042, kettle, delivered 12 days ago, wants a replacement'
writer -> checker: 'Dear customer, we are sorry that your kettle broke. A replacement for order 1042 is on its way.'
checker -> program: 'approved'
program -> customer: 'Dear customer, we are sorry that your kettle broke. A replacement for order 1042 is on its way.'
```

A fact can be dropped at each boundary between agents. The Agent
Engineer Course lists this among the weaknesses of a pipeline, because a
mistake early in the line travels forward to the agents after it
(Multi-agent systems: architectures, roles, the orchestration tax, Agent Engineer Course). One agent that read the message and wrote the reply would have
had the address in its context. The pipeline made three model calls and
two handoffs, and it lost a fact on the way. Add a second agent when it
can do something the first can't, such as work with a focused prompt,
its own small set of tools or other permissions (Multi-agent systems: architectures, roles, the orchestration tax, Agent Engineer Course). The next
lesson counts what each arrangement costs in tokens.

## A manager that delegates to workers

Now a model decides who runs when. A manager model gets the customer's
message, and it can call two workers, `lookup` and `writer`, the way an
agent calls tools. The loop around the manager runs each worker call the
manager asks for, and gives the result back to the manager. This
arrangement has no checker, and the manager's last reply goes to the
customer. Here is the fake manager.

```python
def manager(history: list) -> dict:
    last = history[-1]
    if last["from"] == "customer":
        return {"calls": [{"worker": "lookup", "task": last["text"]}]}
    if last["from"] == "lookup":
        # Like a real model, it passes on what it got, an error message included.
        brief = last["text"] if last["ok"] else last["error"]
        return {"calls": [{"worker": "writer", "task": brief}]}
    return {"answer": last["text"]}
```

Each reply either asks for worker calls or gives the answer. The loop
runs a worker the way a harness runs a tool, and turns its result into a
dictionary with `ok` and either `text` or `error`.

```python
for _ in range(max_turns):
    reply = model(history)
    problem = check_reply(reply)
    if problem:
        stop(problem)
        return
    if "answer" in reply or not reply.get("calls"):
        show("manager", "customer", reply.get("answer", ""))
        return
    for call in reply["calls"]:
        show("manager", call["worker"], call["task"])
        result = run_worker(workers, call["worker"], call["task"])
        show(call["worker"], "manager", result)
        history.append({"from": call["worker"], **result})
        problem = check_result(call["worker"], result)
        if problem:
            stop(problem)
            return
```

`check_reply` and `check_result` are the checks at the two boundaries,
and for now neither of them stops a run. The `manager` step runs the
same message.

#### Example: The manager and its workers

Run this, and compare what you see with the output below.

```sh
python3 agent.py manager
```

Output:

```text
manager -> lookup: 'My kettle from order 1042 stopped working. Please send the replacement to my office address this time.'
lookup -> manager: {'ok': True, 'text': 'order 1042, kettle, delivered 12 days ago, wants a replacement'}
manager -> writer: 'order 1042, kettle, delivered 12 days ago, wants a replacement'
writer -> manager: {'ok': True, 'text': 'Dear customer, we are sorry that your kettle broke. A replacement for order 1042 is on its way.'}
manager -> customer: 'Dear customer, we are sorry that your kettle broke. A replacement for order 1042 is on its way.'
```

The customer gets the same reply, and the address is still lost. This
time a model decided who ran when. Each line that starts with
`manager ->` comes from a model call that picked the next agent, and a
real manager could pick another order, skip a worker or call one twice.
So orchestration by a model can handle a task whose steps nobody wrote
down in advance. In exchange, the same input can take a different path,
which is harder to debug, and each decision is one more model call
(Orchestrators: code- versus model-driven, patterns, anti-patterns, Agent Engineer Course).

Claude Code uses this arrangement for its subagents. A subagent that
gets a task from the main conversation begins with a new context window
of its own. The messages exchanged so far aren't in it. When it
finishes, the main conversation gets back a short summary instead of
everything the subagent read (Create custom subagents, Claude Code documentation). The Academy
course on subagents gives this as a reason to delegate: the main
conversation holds less text and doesn't drift from its topic
(Introduction to subagents, Claude Academy). It also means that a fact from the
conversation that the task text doesn't mention is out of the subagent's
reach. The task text is the handoff, as the brief was in the pipeline.

#### Checkpoint: Who decides the order?

How should the team coordinate the three agents?

- A manager model that decides for each claim which agent runs next, so it can adapt to odd claims.
- A program that runs the three agents in a fixed order and passes each output to the next one.
- Three peer agents in one shared conversation that agree among themselves who goes next.
- A manager model with the three agents as tools and a list of the order in its prompt.

## Sequential, hierarchical and collaborative

The pipeline and the manager are two of the common arrangements. The
Agent Engineer Course describes them with two more, collaborative and
competitive (Multi-agent systems: architectures, roles, the orchestration tax, Agent Engineer Course), and this lesson covers the collaborative one.

- **Sequential.** The agents form a pipeline, and each one works on the
  output of the one before it. The pipeline above is sequential.
- **Hierarchical.** A manager splits the task, hands the parts to worker
  agents and puts their results together. The manager above is
  hierarchical, with two workers.
- **Collaborative.** Peer agents share a workspace or a conversation,
  with none of them in charge. They build on each other's work until
  they agree on a result.

The coordination gets harder in that order. The course rates it low for
a pipeline, medium for a manager and its workers, and high for a group
of peers, and it lists endless discussion among the risks of the last
(Multi-agent systems: architectures, roles, the orchestration tax, Agent Engineer Course). The arrangements also combine, and a step of a pipeline can
itself be a manager with workers. The course advises starting with one
agent, then a pipeline if the task has stages, and a system of many
agents only when one agent has shown that it can't do the task
(Orchestrators: code- versus model-driven, patterns, anti-patterns, Agent Engineer Course).

#### Checkpoint: Name the arrangement

Buckets:

- Sequential
- Hierarchical
- Collaborative

Items:

- A program sends each scanned letter to an extractor, then a classifier, then a filing agent
- A lead agent splits a research question and hands each part to a search agent
- Three reviewer agents post findings in one shared thread and reply to each other until they agree
- A coding agent starts one helper for the tests and one for the docs, and merges what they return
- An editing agent always gets the text of the drafting agent and passes its edit to a layout agent
- A planning agent and a critic agent each edit a shared document whenever they see a need, and neither assigns work to the other

## A fault at each boundary

The manager version has two boundaries: the manager's reply to the loop,
and a worker's result to the manager. We break each one. At the first,
the manager's reply asks for no worker and doesn't answer either, which
is an empty plan. At the second, the order service times out, and the
`lookup` worker returns an error. Here is the empty plan.

#### Example: An empty plan from the manager

Run this, and compare what you see with the output below.

```sh
python3 agent.py empty_plan
```

Output:

```text
manager -> customer: ''
```

The loop takes a reply with no calls as the end of the run, and the
customer gets an empty message. Here is the timeout.

#### Example: A tool error from a worker

Run this, and compare what you see with the output below.

```sh
python3 agent.py tool_error
```

Output:

```text
manager -> lookup: 'My kettle from order 1042 stopped working. Please send the replacement to my office address this time.'
lookup -> manager: {'ok': False, 'error': 'order service timed out'}
manager -> writer: 'order service timed out'
writer -> manager: {'ok': True, 'text': 'Dear customer, we are sorry about the problem. We will get back to you soon.'}
manager -> customer: 'Dear customer, we are sorry about the problem. We will get back to you soon.'
```

The manager passed the error text to the writer as if it were a brief,
and the customer got a polite reply that promises nothing. Nobody in the
shop knows that the lookup failed. Each run ends with a message to the
customer, and neither message is true to what happened. The course asks
for a check of each agent's output before the next agent gets it, so
that a bad result doesn't travel on (Orchestrators: code- versus model-driven, patterns, anti-patterns, Agent Engineer Course). The guarded run passes
`stop_on_error` as the check for worker results.

```python
def stop_on_error(worker: str, result: dict) -> Optional[str]:
    """Stops the run when a worker returns an error."""
    if not result["ok"]:
        return f"{worker} failed: {result['error']}"
    return None
```

When a check returns a text, `stop` prints it after `stopped: ` and then
prints `nothing was sent to the customer`. The check runs in the loop,
after the worker's result is shown and before the manager's next turn.

#### Checkpoint: Predict the report

The `lookup` worker times out again, and this time the loop checks each
worker result. What does the run print? Predict the output, then run it.

```sh
python3 agent.py guarded
```

The run stops at the boundary where the fault happened, and the report
names the worker and the error. A person or a retry policy can act on
that. The writer never ran, and the customer got nothing false.

#### Pitfall: A manager model for steps that never change

A team builds a manager model to run a workflow of four steps that always
run in the same order, because a manager with workers looked more
capable. The manager makes a model call before each step to choose the
step the team already knew, and one day it skips one. The trace shows a
choice at every turn, and none of them was needed. Rule: when the steps
and their order are known in advance, write the order in code, and keep
a model for the parts where the next step depends on what the last one
found (Orchestrators: code- versus model-driven, patterns, anti-patterns, Agent Engineer Course).

The guarded loop has a check for worker results, and its check for the
manager's reply, `stop_on_empty_plan`, lets every reply through. The
`exercise` step runs both faults with the guarded loop.

#### Example: Both faults before the exercise

Run this, and compare what you see with the output below.

```sh
python3 agent.py exercise
```

Output:

```text
fault: empty plan from the manager
manager -> customer: ''
fault: tool error from the lookup worker
manager -> lookup: 'My kettle from order 1042 stopped working. Please send the replacement to my office address this time.'
lookup -> manager: {'ok': False, 'error': 'order service timed out'}
stopped: lookup failed: order service timed out
nothing was sent to the customer
```

## Exercise

In your copy of the `orchestrating-agents` folder, write
`stop_on_empty_plan` so that it returns a text when the manager's reply
has no worker calls and no answer, and `None` otherwise. Run
`python3 agent.py exercise`. The output is the two traces, one per
fault. Writing the check yourself shows that the boundary checks belong
to the code around the model, whoever decides the order.

A good result: the empty plan trace ends with a `stopped:` line that
says the manager returned an empty plan, then
`nothing was sent to the customer`, and the tool error trace is as it
was. Compare with `python3 model_answer.py` in the same folder. Which other
boundary in this arrangement could pass a bad result on, and what would
its check look for? The opening comment of `model_answer.py` gives one
answer.

Stretch: Add a check for the writer's result as well: stop when the draft doesn't name the order the customer wrote about. Pass it as check_result in step_tool_error, run tool_error and see which boundary now catches the fault.

## Recap

1. In orchestration by code, a program decides which agent runs when.
   It runs the same way every time and costs no model call to decide
   (Orchestrators: code- versus model-driven, patterns, anti-patterns, Agent Engineer Course).
2. In orchestration by a model, a manager decides and calls the other
   agents as tools. It can handle steps nobody planned, and the same
   input can take another path (Orchestrators: code- versus model-driven, patterns, anti-patterns, Agent Engineer Course).
3. A fact can be lost at each handoff. Add a second agent when it can
   do something the first can't (Multi-agent systems: architectures, roles, the orchestration tax, Agent Engineer Course).
4. Sequential, hierarchical and collaborative arrangements get harder to
   coordinate in that order (Multi-agent systems: architectures, roles, the orchestration tax, Agent Engineer Course). Check the result at each boundary
   in code, and stop with a report when it is wrong.

#### Habit

The next time you hand a task to a subagent or a second agent, read the
task text it gets and check that it holds every fact the work depends on.

## References

- Orchestrators: code- versus model-driven, patterns, anti-patterns, Agent Engineer Course, https://lsimons.github.io/agent-engineer-course/18-orchestrators/
- Multi-agent systems: architectures, roles, the orchestration tax, Agent Engineer Course, https://lsimons.github.io/agent-engineer-course/07-multi-agent-systems/
- Create custom subagents, Claude Code documentation, https://code.claude.com/docs/en/sub-agents
- Introduction to subagents, Claude Academy, https://academy.claude.com/courses/introduction-to-subagents