# Chat, agent or automation

> Pick chat, agent or automation for a task by its worst case, match the checks to the level, and write the rule that would move the pick up or back.

Page: https://schubergphilis.github.io/ai-training/using-agents/chat-agent-or-automation/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

In this lesson we pick how to apply a model to a task. Chat, agent, and
automation differ in how much the
model does between one moment you look and the next, and the lesson gives
you a way to pick the level for a task and to write down what would change
the pick. You
don't run anything here. The tasks are supplied, and the work is reading
them and deciding.

The concepts course gave you a test for telling an assistant from an
agent, and the safety course showed where a person has to approve before
an agent acts. This lesson puts the two together into one decision you
make before the work starts: how much of it the model does on its own.

## Chat, agent, and automation

The same model can be used in three ways, and the difference is where you
are while it works.

**Chat.** You type, the model answers, you read the answer and type again.
Every step is yours, and the model does one step per turn. A difficult
email is a chat task. You know what you want to say and the words won't
come. You paste the thread and ask for a first draft. You cut a paragraph
and ask for a softer second sentence, and you send the version you end up
with. Every turn is a decision about tone, and you are the one making it.
Chat fits thinking and one-off drafts, and any other task where you would
change your mind halfway through.

**Agent.** You write a brief, and the agent works through the steps with
its tools. It checks in with you at the points the brief names. A folder
of two hundred misnamed scans is an agent task. You say what the names
should look like and what to do with a file it can't read. You also say
that it reports back before deleting anything. Then it works, and you read
the report. The steps are known, but the path through them varies per file,
and a person doesn't need to see each one. An agent fits a bounded task
with clear done-criteria, where you want to read the result once instead
of steering every step.

**Automation.** An event starts the run while nobody is watching, the
model does its part, and the result goes where it goes. Invoices renamed the moment they arrive
in a mailbox are an automation task. The steps and the path are both
fixed, the same thing happens every time, and the work is checked by
sampling later, or by whoever opens the folder. Automation fits
repetitive, low-stakes work that has been tested enough that nobody needs
to watch it.

The pattern across the three is one variable moving. In chat the model
does one step and waits. In an agent it does many steps and waits at the
points you named. In automation it does the whole task and waits for
nobody. The word for that variable is autonomy, and each level trades your
attention for the model's independence. A wrong pick either wastes your
attention, when you steer step by step through work that has one right
path, or removes your attention from the place where it was needed.

#### Checkpoint: Which level fits each task?

Buckets:

- Chat
- Agent
- Automation

Items:

- Finding the right tone for a reply to one angry customer, when you are not sure yet what you want to offer
- Renaming two hundred scanned documents to one naming pattern and reporting the ones it could not read
- Adding the amount and the sender of each incoming invoice to a spreadsheet as it arrives
- Deciding how to structure a proposal you have not thought through yet
- Collecting the figures for the monthly report from four shared folders into one draft you will read

## Autonomy rises and the checks rise with it

Moving up a level gives the model more steps between checks. The trade
only holds when the checks on the other side grow with it. In chat, the
check is you reading every answer, and you need nothing else. In an
agent, you read the result once. The brief has to contain the checks you
won't be there for, such as what the agent must not do and where it stops
to ask. In automation, you don't read anything at the
time, and the checks have to exist and be tested before the first run. A
sample you look at each week is one. An error the system reports instead
of guessing through is another, and so is a way to undo a batch. A step-up in autonomy without
the matching step-up in checks is a task that runs unwatched and
unchecked, and the failures that follow are the ones the safety course
described: the mistake now has hands
(Building effective human-agent teams, Claude Academy).

The level is also picked for the task's worst case, and this is the part
people get wrong. A support queue looks like an automation task. Most
tickets are a password reset, a delivery question, or a refund under the
limit, and a well-briefed agent answers those correctly all day. One ticket
in fifty is different. It is a complaint that mentions a lawyer, or a
customer who was told something wrong by a person last week. The forty-nine routine tickets make the case for
automation, and the fiftieth decides the level. If a wrong answer on that
one ticket is expensive, and the system can't tell that ticket from the
others with certainty, the queue stays at the agent level, with a person
approving the result. Call that "agent with approval": the agent drafts
every reply and a person sends it, or the agent sends the routine ones and
routes anything it isn't sure about to a person. It is the agent level with
the check that level needs, and the count of levels stays three. The level is
set by the ticket you can't afford to get wrong, and the routine tickets
only decide how little attention that level should cost.

So the question for a task is never "what does this task need on most
days?" It is "what does this task need on its worst day, and can the system
tell that day from the others?" A task with a worst case that's cheap to fix,
or that the system reliably detects and hands over, can run at the level
its routine case suggests. A task whose worst case is expensive and looks
like the routine case from the outside is kept one level down.

#### Checkpoint: Which level does the queue get?

Which level does the support queue get, and why?

- Automation, because the agent answers forty-nine of fifty tickets well and a person can review the complaints when they come up.
- Chat, because the fiftieth ticket is too important to hand to an agent, so a person should steer every reply.
- Agent with approval, because the worst ticket is expensive to get wrong and the agent cannot tell it apart, so a person sends every reply the agent drafts.

## The moving rule

A pick made this way has a reason, and a reason can be written down. The
useful form is two sentences per task, one for what would move the task
up a level and one for what would move it back down. "The invoice renaming
goes to automation once a month of agent runs hasn't needed a correction and
the folder is backed up nightly. It comes back to agent if a run renames a
file we can't match to an invoice." A rule like that makes the pick
reviewable by someone who wasn't there when it was made, and it makes the
next change a decision that someone takes on purpose.

What moves a task up is evidence, and what moves it back is a new worst
case. Evidence is a run of results without corrections, a test on the
cases that used to fail, a backup or an undo that turns the worst case into
a cheap one, or a check that finds the odd case and routes it to a person.
A new worst case is a task that starts to receive a kind of input it
didn't before, or a wrong result that got more expensive because the folder
is now shared with a client. A correction that nobody caught for a week is
a new worst case too. So is a changed rule, for example a new customer
contract that says a person answers every message. When the level is written with its moving rules, an
improvement in the tool is one more piece of evidence and a change in the
task is a reason to look again, and both are decisions someone made on
purpose (AI Fluency for builders, Claude Academy).

Writing the rule also stops the two silent moves. In the slide up, a task
at the agent level with approval becomes automation because the person
approving stopped reading and started clicking, and nobody decided
anything. In the slide down, a task that could safely run unattended stays
in chat because it always has, and a person spends an hour a week steering
a task with one right path. A written rule names the
evidence the first move needs and the worst case that would justify the
second, and a task with neither stays where it is.

#### Checkpoint: Six months of approvals

The operations lead has approved the agent's monthly report draft for six
months, and admits that since about March the approval has been a click
without a read. Nothing in the drafts has been wrong. What do you do?

- Move the report to automation, since six months of approvals with no changes is the evidence the lesson asked for.
- Keep the approval step as it is, because a person clicking approve each month is a check.
- Bring the report back to chat and have the lead build it turn by turn, because approvals that nobody reads show the task was never safe to delegate.

#### Pitfall: Routine nine times, consequential the tenth

A task that goes the same way nine times out of ten is the one most
likely to be picked for the wrong level, because the nine times are all
anyone sees when the pick is made. The renaming and the support reply
both look like a fixed path, and the pick is automation. The tenth time is
the scan that turns out to be a signed contract, or the ticket that
mentions a lawyer. Pick the level for the tenth time, and use
the nine to decide how cheap that level should be. When the tenth time
can't be told from the others by the system, the level is the one where a
person still sees each result.

## Exercise

For each of the five tasks below, pick chat, agent or automation for the
task's worst case, and write one sentence for what would move the task up
a level and one for what would move it back. Fifteen lines, and fifteen
minutes is enough.

```text
1. Answering the questions new colleagues ask in the team channel
   about where things are and how the tools work.
2. Turning the notes of a customer meeting into a follow-up email
   to the customer.
3. Sorting the receipts that arrive in the expenses mailbox into
   the month and the category they belong to.
4. Writing the first version of a project plan for something the
   team has never done before.
5. Checking each new supplier contract against the company's
   standard terms and listing the differences.
```

A good answer has the plan in chat, because the thinking is the task and
you would change course halfway. It has no up move while the plan is being
thought out, and once the plan exists, filling in the sections from notes
is a different task that an agent can do. It has no down move, because chat
is the lowest level. The follow-up email is at agent with you sending it,
because the meeting is in your memory and the customer receives the result.
It moves up to automation only when the notes are complete enough to draft
from without you and a wrong email costs little, and it moves back to chat
when a meeting was difficult and the wording is the work. The receipts are
at automation with a monthly sample check and a way to move a wrong file.
They moved up on a month of agent runs without a correction, and they come
back to agent if a receipt turns out to be a contract or an invoice above a
set amount. The team-channel questions are at agent. The agent answers from
the team's own pages and says when it can't find an answer. It moves up to
automation once a month of answers has been checked and none was about
access or a password, and it moves back to chat if a wrong answer about
access or a password shows up. The contract check is at agent with a person
reading each list, because a missed clause is the tenth time. It moves up
for the standard contracts only, once a test on last year's contracts has
found every difference a lawyer found, and it moves back to agent for all
contracts if a supplier sends a form the test never saw. Where your answer differs, the sentence that matters is the one
about the worst case, so check that one first. Which of your five picks
would slide up on its own if nobody wrote the rule?

Stretch: Take a task from your own work that a model already does or could do, write its level with the two moving sentences, and check the pick against the task's worst day of the last year.

## Recap

1. Chat, agent and automation are three ways to apply one model, and the difference is how many steps it does before you look.
2. Autonomy rises across the three. The checks rise with it, from your reading in chat to the brief's stop points for an agent to a tested sample and undo for automation (Building effective human-agent teams, Claude Academy).
3. The level is picked for the task's worst case. A worst case that is expensive and can't be told from the routine case keeps the task one level down.
4. Each pick gets a moving rule: the evidence that moves the task up, and the new worst case that moves it back (AI Fluency for builders, Claude Academy).
5. The written rule stops the two silent moves, the approval that becomes a click and the task that stays in chat because it always has.

## References

- Building effective human-agent teams, Claude Academy, https://academy.claude.com/courses/building-effective-human-agent-teams
- AI Fluency for builders, Claude Academy, https://academy.claude.com/courses/ai-fluency-for-builders