Workflows before agents

In this lesson we run three fixed arrangements of model calls for a small shop that answers customer emails. In each one the code decides which call comes next, and the model only writes the text of each call. After that we look at a kind of task that fixed code can’t hold, and that needs an agent that chooses its own steps.

In Stopping the loop on purpose the model chose every tool call, and the harness rules decided when the run ended. Here the order of the calls is written in the program before the first email arrives. The model calls are fake: functions written by hand that read a prompt name and some text and return text. Every run prints the same result. The complete program is site/examples/building-agents/workflows-before-agents/workflows.py in the repository. Copy the workflows-before-agents folder, and run each step below inside your copy as python3 workflows.py <step>. No application programming interface (API) key is needed.

Who decides the next step

Anthropic’s engineering article on building agents sorts systems that use models into two kinds [1]. In a workflow, the path through the model calls and the tools is set in code that the developers wrote. In an agent, the model directs the process itself and picks the tools it uses as it goes. The article advises starting with the simplest thing that works. Often that’s one model call, prepared well, with retrieved text and a few examples in the prompt. Add steps only when the simpler version doesn’t do the job well enough, because each step adds time and cost [1].

Between one call and an agent are the workflow patterns: fixed arrangements of model calls, inside one task, where code controls the order. This lesson runs a chain, a route, and parallel calls. A fourth, a draft-and-critique loop, gets its own lesson later in this part. When the parts of the task are whole agents that hand work to each other, the design is a multi-agent system. The lesson Orchestration in code or by a model opens the part of the course that covers it.

A chain with a gate

A chain runs model calls one after another, and each call works on what the call before it returned. Code can check the output between two calls and stop the chain when the check fails [1]. The article recommends a chain for a task that breaks into the same steps every time. Each call then has a smaller job and is more likely to get it right, and the price is the time of the extra calls [1].

The shop’s chain has two calls. extract reads the email and returns the order number and what the customer wants. The gate is plain code. It looks the order up. When it finds the order, write writes the reply. Any other email goes to a person.

def chain(email_id: str) -> None:
    calls.clear()
    print(f"chain, email {email_id}")
    extracted = model("extract", EMAILS[email_id])
    print(f"  call 1 extract: {extracted}")
    facts = gate(extracted)
    if not facts:
        print("  gate: no known order, the email goes to a person")
    else:
        print(f"  gate: order {facts.split()[0]} found")
        print(f"  call 2 write: {model('write', facts)}")
    print(f"  model calls: {len(calls)}")

Email a1 names order 1042. Email a2 asks for a replacement toaster and doesn’t name an order.

Example · run it

Two emails through the chain

Run this, and compare what you see with the output below.

python3 workflows.py chain

Output

chain, email a1
  call 1 extract: order=1042 wants=replacement
  gate: order 1042 found
  call 2 write: Dear customer, a replacement for the kettle of order 1042 is on its way.
  model calls: 2
chain, email a2
  call 1 extract: order=none wants=replacement
  gate: no known order, the email goes to a person
  model calls: 1

Output verified in CI from site/examples/building-agents/workflows-before-agents/chain.py.

The second email stops after one call. Without the gate, write would get order=none and could write a reply about an order that doesn’t exist. The steps are the same for every email, so the program can hold them, and the gate is where the program checks the model’s work.

A route to a prompt per class

A route has one call that sorts the input into a class, and code then sends the input to the prompt or the process for that class. The article recommends it when the inputs come in distinct kinds, and each kind gets better answers from its own prompt. With one prompt for every kind, a change that helps one kind of input can make the answers to another kind worse [1]. A route can also send easy inputs to a small and cheap model, and hard ones to a stronger model [1].

The shop’s classify call returns refund, repair, question or other. Each of the first three has its own prompt. The refund prompt knows the 30-day limit, and other has no prompt, so code sends it to a person.

for email_id in ["b1", "b2", "b3", "b4"]:
    label = model("classify", EMAILS[email_id])
    if label in ROUTES:
        answer = model(ROUTES[label], EMAILS[email_id])
    else:
        answer = "(no prompt for this class, the email goes to a person)"
    print(f"  {email_id} {label:<8} {answer}")

Example · run it

Four emails through the route

Run this, and compare what you see with the output below.

python3 workflows.py route

Output

route, four emails
  b1 refund   Your refund for order 1042 is on its way.
  b2 refund   Order 1131 was delivered 41 days ago, past the 30-day limit.
  b3 question Descale the kettle with vinegar once a month.
  b4 other    (no prompt for this class, the email goes to a person)
  model calls: 7

Output verified in CI from site/examples/building-agents/workflows-before-agents/route.py.

The run makes 7 model calls for four emails, one classify call each and one more for each of the three emails that have a prompt. When the classifier picks the wrong class, the email goes to a prompt that was never written for it. Make the classes clear, and give the emails that fit none of them a class of their own.

Parallel calls

Parallel calls come in two forms [1]. Sectioning splits the work into independent parts and runs them at once. For example, one call writes the reply while another checks the email for something a person must handle. Voting asks one question several times and combines the answers. A judgment that one call can get wrong becomes more reliable that way. The article recommends parallel calls for parts that can run side by side to save time, or for a judgment that needs more than one attempt to trust [1].

In the shop, email c1 mentions a card number, and the screen call runs next to the reply call. Email c2 doesn’t say whether the customer wants a refund or a replacement. Three vote calls answer that question, and the shop acts on the answer that most of them give. The fixture counts 2 seconds per model call. The seconds are an assumption written by hand, to compare one way of running the calls with the other.

Example · run it

Sections and votes

Run this, and compare what you see with the output below.

python3 workflows.py parallel

Output

parallel, at 2 s per model call
  sections, email c1
    reply:  Dear customer, a new kettle for order 1042 is on its way.
    screen: flag: card number
    the reply waits until a person removes the card number from the email
    model calls: 2, 4 s one after another, 2 s at once
  votes, email c2, refund or replacement?
    votes: refund replacement refund
    majority: refund, 2 of 3
    model calls: 3, 6 s one after another, 2 s at once

Output verified in CI from site/examples/building-agents/workflows-before-agents/parallel.py.

The votes disagree, refund replacement refund, and that’s the reason to ask more than once. One call could have returned replacement. The votes cost three times the tokens of one call. When they run at once, the customer waits for one call, 2 seconds instead of 6. The code still decides which calls run, how many votes there are, and what counts as a majority.

Checkpoint · match

Which arrangement fits the task?

Match each task to the arrangement that fits it.

For each new product, write a description and then translate the description into German.Choose…A chainA routeParallel callsAn agent that chooses its steps

Send each support email to the prompt for billing, for delivery or for returns.Choose…A chainA routeParallel callsAn agent that chooses its steps

Check a contract for four kinds of risky clause, with one call per kind, and join the findings.Choose…A chainA routeParallel callsAn agent that chooses its steps

Find why one customer sees the wrong price, where each record you read decides which record to read next.Choose…A chainA routeParallel callsAn agent that chooses its steps

CheckHintSkip

When the task needs an agent

Each pattern so far has a path that you can draw before the first email arrives, branches included. The route picks one of four paths, and the chain either stops at the gate or goes on. Each of those paths is in the code. An agent fits a task where you can’t draw the paths. Think of a search for the cause of a fault: each finding opens new places to look, and the count of steps is unknown at the start. The article keeps agents for tasks like that, where nobody knows the number of steps or the path before the work begins [1]. An agent is the loop from the earlier lessons: the model calls tools, reads what they return, and decides the next step until it is done or a stop rule ends the run.

That freedom has a price. An agent costs more to run, and a wrong early step can make the steps after it go wrong too. The article advises testing it a lot in an environment where its mistakes do no harm, with limits on what it can do [1]. So the question for a new task is whether you can list its possible steps today and draw the paths between them. If you can, write them as code and call the model inside each step. If nobody can list them before the task starts, you need an agent, and its stop rules from the earlier lesson.

Checkpoint · choice

What makes a task need an agent?

A developer has to decide between a workflow and an agent for a new task. Which property of the task means that fixed code can’t hold its steps?

The task needs four or more model calls before it can give the answer.The task calls a tool, such as an order lookup, along the way.Nobody can list the possible next steps before the task starts.A wrong answer would cost the shop a lot of money.

CheckHintSkip

Pitfall: A chain without a gate

A team chains two calls: one reads the customer’s email and pulls out the order number, and the next writes the reply from it. The code passes the first output to the second call without a check, so on an email without an order number the first call returns none and the second writes a confident reply about a replacement for an order that doesn’t exist. Put a check in code between the calls, and send what fails the check to a person, the way the shop’s gate did with email a2.

Exercise

For each task below, decide whether fixed code can hold its model calls as a chain, a route or parallel calls. If it can’t, the task needs an agent that chooses its own steps. Write one or two sentences per task, and name the property of the task that decides it. Ten minutes is enough. Naming that property before you build anything keeps you from building an agent that a few lines of code could replace.

1. Each night, summarize the day’s support emails, and then turn the summary into a list of the most common problems.
2. A help desk gets questions in English, Dutch and French, and a native speaker has tuned one prompt for each language.
3. Grade one essay on spelling, on structure and on its argument, with one call for each criterion.
4. Make a failing test pass in a code base: read the error, open the files it points to, change the code and run the tests again until they pass.

A good result calls task one a chain, because the second call works on the output of the first and the steps never change. Task two is a route, because the language of the question decides which prompt answers it. Task three is parallel calls of the sectioning kind, because the three criteria don’t depend on each other. Task four needs an agent, because nobody can list the files to open or the changes to try before the first test run, and each run opens new places to look. Which of the first three would you move to an agent if its inputs changed, and what change would it take?

Stretch: Add an email to EMAILS that the classify call sends to repair, add it to the list in step_route, and run python3 workflows.py route. Then change VOTES so that replacement wins, and run the parallel step again.

Recap

1. In a workflow, code decides the path of the model calls. In an agent, the model decides its own steps and tools [1].
2. Start with the simplest version that works, often one model call, and add calls only when it doesn’t do the job well enough [1].
3. A chain runs calls in a fixed order, each on the output of the one before, and code between them can stop it. A route sorts the input and sends it to a prompt per class. Parallel calls split a task into independent sections, or ask one question more than once and act on the answer most calls give [1].
4. A task needs an agent when nobody can list its possible steps or draw its paths before it starts. A branch that code can write down, as in a route, is still a workflow. The agent then costs more and needs stop rules and testing where its mistakes do no harm [1].

You can now

• Picks a design pattern for a task and says why

Mark lesson finishedI know this, skip it

Habit

The next time you put one model call after another, write down what code checks the first call’s output before the second call uses it.

References

1. Erik S., Barry Zhang. Building effective agents. Anthropic Engineering. Reference. Anthropic effective agents