# Checking habits that run when you are in a hurry

> Find the row that does not add up in a table a chat assistant wrote, learn the four checks that run on everything that leaves your desk, and set how deep each one goes from the cost of being wrong.

Page: https://schubergphilis.github.io/ai-training/safety/checking-habits/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

In this lesson we find the row that doesn't add up in a table a chat
assistant wrote from three months of ticket counts. The check that finds
it is one of four, and the four together are a routine short enough to
run in the ten minutes before a meeting. Earlier lessons in this course showed what
an invented specific looks like and why attention to a good tool fades.
The routine here is what you do about both, and the last section sets how
deep each check goes from the cost of being wrong.

## A table sent in a hurry

It is ten minutes before a meeting. A colleague pastes three monthly
reports into a chat assistant and asks for "a table of tickets closed per
team, with a total per team and a grand total". The assistant answers in a
second, and the table goes into the slides as it came back. The teams and
the numbers are invented for this lesson, and the assistant's table is the
author's.

| Team        | Jul | Aug | Sep | Total |
| ----------- | --- | --- | --- | ----- |
| Design      | 124 | 98  | 111 | 333   |
| Support     | 82  | 86  | 90  | 258   |
| Marketing   | 145 | 162 | 139 | 466   |
| Operations  | 67  | 71  | 69  | 207   |
| Grand total |     |     |     | 1,264 |

The table reads well. The columns line up, the totals are in the right
place, and every number is one your colleague could have typed. One row
adds up wrong. Before reading on, add up the rows yourself and decide which.

#### Checkpoint: Which row does not add up?

Which row's total isn't the sum of its three months?

- Design: 124, 98 and 111 with total 333.
- Support: 82, 86 and 90 with total 258.
- Marketing: 145, 162 and 139 with total 466.
- Operations: 67, 71 and 69 with total 207.

A short program written for this lesson adds every row again and compares
the sum with the total the assistant wrote. This is what it prints.

```text
ok        Design      written   333  recomputed   333
ok        Support     written   258  recomputed   258
MISMATCH  Marketing   written   466  recomputed   446
ok        Operations  written   207  recomputed   207
MISMATCH  Grand total written  1264  recomputed  1244
rows that do not add up: 1 of 4 (Marketing)
```

Marketing closed 446 tickets, and the table says 466. The grand total is
wrong by the same 20, because the assistant added its own row totals
rather than the months. One slip, two wrong numbers on the slide, and
nothing in the way the table looks tells you which two. The model predicted
the totals the way it predicts every other number in the table, and 466
is a likely number. It isn't the sum. The lesson
[What an invented fact looks like](https://schubergphilis.github.io/ai-training/safety/spotting-hallucination/#specificity-is-the-tell)
called this specificity without a source. A total is a specific, and its
source is the arithmetic, which you can redo in the time it takes to read
the row.

## The routine

The check that caught the Marketing row is one of four. Together they're
what **checking habits** means in this course, and they take a few minutes
on a page of output.

1. **Read the output against the request.** Put the prompt next to the
   answer and read them together. Did you ask for tickets closed, and is
   that what the column says? Did you ask for a total per team, and is
   there one for every team? A model answers a nearby question as fluently
   as the one you asked.
2. **Look for claims you didn't supply.** Every number, name, date and
   reference in the output came from what you pasted, or it came from the
   model. Mark the ones you didn't supply. Each is a claim to check or to
   take out, as the lesson on invented facts showed.
3. **Open one link.** If the output cites anything, a page, a document, a
   section number, open one and read whether it says what the output says.
   One is enough to tell you whether the rest deserve the same.
4. **Redo one calculation.** Pick one row, one percentage, one date
   difference, and do it by hand or in a spreadsheet cell. If it holds,
   the others probably hold too. If it fails, none of them can go out
   unchecked.

They're one routine because they run together and in this order on
everything that leaves your desk. Output that stays with you, a
brainstorm or a note to yourself, may skip them, and the next section
says how to tell. For output that goes to someone else, you don't decide
each time whether to run them. Anthropic's AI Fluency course calls the skill of
judging what a model gave you discernment. It pairs that skill with the
skill of asking well, which it calls description, and has you move between
the two in a loop (AI Fluency: Framework and foundations, Claude Academy). The routine here is
discernment made small enough to run when you have ten minutes.

#### Pitfall: The check that gets skipped

The colleague with ten minutes before the meeting ran the first check and
the second. The table answered the request, every monthly count in it was
one from the reports, and the totals were marked as the model's own. The third check had nothing to open, because the
table has no links. The fourth check, redo one calculation, felt like the
one that could wait, because adding up a row takes longer than reading it.
It is the check that finds the error. A busy person skips it, and errors
get through in those busy moments. When you notice
that you are about to skip a check, that is the check you run. A routine
that has a step you drop under time pressure is a routine that only works
when you don't need it.

## Depth scales with stakes

The routine is the same every time, and the depth of each check isn't. The
last lesson,
[When the tool is usually right](https://schubergphilis.github.io/ai-training/safety/when-the-tool-is-usually-right/#build-the-check-in),
said to build the check into the process and to set its depth from what
happens when the output is wrong. Fluency is constant and consequences are
not, so the cost of a mistake is the only thing that should change the
effort. The depth comes from whether the output can be undone, and from
who reads it and whether they check it themselves.

A brainstorm list of names for an internal project isn't checked at all.
A wrong entry doesn't cost anything, and you throw most of them away. A summary of a
meeting for your own notes gets one read against the request, because you
were in the meeting and you'll notice a wrong point when you use the
notes. A briefing with three cited reports for your manager gets every
link opened and every number redone, because your manager acts on it and
does not open the reports. A price table on the public web page gets every
value read by a second person, because it cannot be recalled once a
customer has seen it, and the customer checks nothing. The list, the
notes, the briefing and the price table read as fluently as each other.
The depth of the check comes from the cost of the mistake.

One more case sets the depth to the maximum whatever the size of the task.
A domain you cannot judge is high risk even when the output is one
paragraph, because you cannot see the mistake. A one-paragraph legal clause
goes to the legal team, however well it reads.

#### Checkpoint: How deep does each check go?

Match each output to the depth of checking it gets before it leaves your
desk.

Statements:

- A list of twenty candidate names for an internal project
- A summary of a meeting you attended, for your own notes
- A briefing with three cited reports, for your manager to act on
- A price table for the public web page

Options:

- No check
- One read against the request
- Every link opened and every number redone
- Every value read by a second person

Harvard's CS50 course gives its students an AI helper (CS50's Introduction to Computer Science, Harvard CS50) (Teaching CS50 with AI: Leveraging Generative Artificial Intelligence in Computer Science Education, Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2024), 750-756).
The helper also answers questions on the course forum. A staff member can
endorse, amend or delete each of those answers, and the forum shows which
ones a staff member endorsed, so a student can tell a checked answer from a
raw one (Teaching CS50 with AI: Leveraging Generative Artificial Intelligence in Computer Science Education, Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2024), 750-756). The idea transfers to your desk. What the assistant
wrote and what you have checked are two different things, and the depth of
checking is what turns one into the other. A later lesson in this course is
about making that difference visible to the next reader.

## Exercise

Pick one document from your week that an assistant wrote or drafted: a
summary, a table, a reply, a report section. Run the four checks on it in
a note. Read it against the request you gave and write down any question
it answered instead. List the claims you did not supply. Open one link or
reference, if it has any, and write what the source says next to what the
document says. Redo one calculation, if it has any, and write both numbers
down. Fifteen minutes is enough.

A good result is a note with four headings and something under each, even
if that something is "no links" or "no numbers". Which of the four checks
would you have skipped, and would the document have gone out with what it
found?

Stretch: Do the same for a second document, one going to a reader outside your team, and compare how deep each check went.

## Recap

1. Before a model's output leaves your desk, read it against the request and look for claims you did not supply. Then open one link and redo one calculation.
2. A total is a specific like any other, and its source is the arithmetic. One wrong row total in a table can put two wrong numbers on a slide, and the table looks the same either way.
3. The check you are about to skip is the one you run. Errors get through at the moment a routine loses a step, which is when you are busy.
4. The routine is the same every time, and its depth comes from the cost of being wrong: whether the output can be undone, and whether its readers check it themselves.
5. A habit is better than a rule because it runs when you are in a hurry, which is when a rule gets skipped.

## References

- AI Fluency: Framework and foundations, Claude Academy, https://academy.claude.com/courses/ai-fluency-framework-foundations
- CS50's Introduction to Computer Science, Harvard CS50, https://cs50.harvard.edu/x/
- Teaching CS50 with AI: Leveraging Generative Artificial Intelligence in Computer Science Education, Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2024), 750-756, https://doi.org/10.1145/3626252.3630938