A prototype you throw away

In Splitting the work into components the agent can build one at a time you wrote the acceptance test for the import parser, and that test treats a line that starts with x as a done item. The rule came from the colleague who asked for the feature. They remember their old app writing x , and nobody has looked at the file. In this lesson we check that rule before the parser is built. We write down the one question, have the agent build the smallest program that answers it, put the answer in the spec and delete the program.

Let the agent interview you ended with open questions in the spec, and some of them can’t be settled by talking. Whether a file uses one done mark or two is a fact about the file, and a short program finds it out faster than a meeting does. Thomas and Hunt call this a prototype built to learn: it answers a question, it can leave out completeness and robustness, and it is thrown away when the question is answered [1]. With a coding agent, such a program takes little time to write, so trying the idea often costs less than discussing it.

The fixture

The fixture is in the course repository under site/examples/coding-with-agents/prototype-to-throw-away/. Copy that whole directory to a place of its own. The copy holds fixture-repo/, where the agent works, and a sample/ directory with a script that the second half of the lesson uses. fixture-repo/ holds SPEC.md, the spec for the import command of the to-do program, and old-list.txt, a copy of the colleague’s old file. The other files are for the course’s own checks. Change into fixture-repo for every step until the Exercise, except the test change, which is in your copy of an earlier fixture. The reset is to delete the copy and copy again.

The spec ends with this section:

## Open questions

- Which done marks the old file uses. The colleague thinks it is `x` and a
  space, but nobody has looked. `old-list.txt` is a copy of the file.

If the answer is “only x ”, the parser and its test don’t need a change. A second mark in the file would make the parser import those items as open. The test would still pass, because none of its input lines has that mark.

One question before any code

A throwaway prototype is a small program built only to answer one design question, and deleted once it has. The question comes first, written in one sentence, before you ask for any code. Without it, the agent builds whatever seems useful: a parser, then a test for the parser, then a command that calls it. That’s a first version of the import, built without the components and checks that you designed in Splitting the work into components the agent can build one at a time.

A prototype can answer a question when the program prints the answer, such as a count or two outputs side by side. “Which done marks does old-list.txt use?” is such a question. “Should we build the import at all?” isn’t, because no program prints the answer. “How should the import work?” isn’t either, because it asks for the whole design.

Checkpoint · choice

Which question fits a prototype?

Which of these questions should the engineer give the agent as the one question for a throwaway prototype?

Whether the import should be built at all, or the colleague could type the items by hand.How the import should be designed, from the parser to the report it prints at the end.Whether the old file marks done items in any other way than with an x and a space.Whether the old file will cause any kind of problem for the import once it is built.

CheckHintSkip

The smallest program that answers it

Start your coding agent in fixture-repo and give it the question, and say what the program reads and prints. Say what to leave out, too. Tests, error handling and structure are effort spent on code that you delete, and a brief that doesn’t mention them leaves that choice to the agent. Say also that it may create one file and change nothing else.

Read SPEC.md. Its open question is which done marks old-list.txt uses.
Write a throwaway script, count_forms.py, that answers only that. It reads
old-list.txt (hard-code the name) and prints how many lines start with
each first word, most common first, the top five.
No tests, no error handling, no command-line options. Create only
count_forms.py, and don't start on the import.

Counting first words is enough. A done mark is the first word of its line, and a mark that the colleague forgot shows up near the top of the list. The course ships one version of such a script in sample/count_forms.py, so the output below is checked on every build:

# Throwaway: answers which done marks old-list.txt uses. Delete after.
from collections import Counter
from pathlib import Path

lines = Path("old-list.txt").read_text(encoding="utf-8").splitlines()
words = Counter(line.split()[0] for line in lines if line.strip())
for word, count in words.most_common(5):
    print(f"{word}: {count}")

The file name is hard-coded, and the whole file is read at once. It has no functions or tests. In the parser each of those would be a defect, and for one run on a short file none of them matters.

Example · run it

Run the course's prototype

Run this in fixture-repo, and compare what you see with the output below.

python3 ../sample/count_forms.py

Output

x: 10
[x]: 4
Buy: 3
Book: 2
Call: 2

Output verified in CI from site/examples/coding-with-agents/prototype-to-throw-away/sample_run.py.

Run the script your agent wrote in the same directory, with python3 count_forms.py. Its format may differ, and the counts for x and [x] should be the same. If they differ, one of the two scripts is wrong, and reading both against old-list.txt tells you which.

Checkpoint · multi-choice

What does the prototype brief ask for?

Which two of these belong in the brief for the throwaway script?

Select exactly 2.

Hard-code the name of the one file the script reads.Add unit tests for the counting in the script, so the numbers can be trusted.Print the counts and nothing that the question doesn't need.Put the counting in importer.py, so the parser can reuse it.Catch a missing or unreadable file and print a clear message.

CheckHintSkip

Write the answer down, then delete the program

The output answers the question. The file has two done marks: x starts ten lines and [x] starts four. A parser built from the spec as it stands would import four done items as open. The prototype found the fact, and the decision is still yours. Here the decision is that [x] marks a done item too. Write it into the behavior and take the question out of the open questions:

2. A line that starts with `x` or `[x]` and a space is a done item, and
   the mark isn't part of its text. The old file uses both (10 and 4
   lines).

The acceptance test changes with the spec. It is the test_importer.py that you wrote in your copy of the fixture of Splitting the work into components the agent can build one at a time. Give it one more input line and its expected item:

    def test_parse_lines_keeps_text_and_done_mark_and_skips_blank_lines(self):
        lines = [
            "Buy milk",
            "",
            "x Call the plumber",
            "  Water the plants  ",
            "[x] Pay the bill",
        ]
        self.assertEqual(
            importer.parse_lines(lines),
            [
                {"text": "Buy milk", "done": False},
                {"text": "Call the plumber", "done": True},
                {"text": "Water the plants", "done": False},
                {"text": "Pay the bill", "done": True},
            ],
        )

A parser that knows only x returns the last line as an open item with the text [x] Pay the bill, so the test fails on it. You change the test before the parser is briefed, and the brief’s “Do not change the test” still holds for the agent. Then delete both scripts, from fixture-repo:

rm count_forms.py ../sample/count_forms.py

The answer is in the spec, where a fresh session and a reviewer read it. The code that found the answer is gone, so nobody can build on it. In an interview with Venners, Hunt and Thomas compare a prototype to the front of a film set, with no building behind it. They say it is made to be thrown away, and that the code a system keeps is a different kind, which they call tracer code [2].

Pitfall: The prototype that became the parser

An engineer had an agent write a quick script to see how an old file marked its done items. The script worked, so the engineer asked the agent to add the import command around it. The script had a hard-coded path and no tests, and it was now the parser. A month later a colleague ran the import on their own laptop, and it stopped with an error about a path that existed only on the engineer’s laptop. The rule: a prototype answers its question and is deleted, and the parser is built from the spec, with its own test.

Checkpoint · scenario

The script works. Now what?

The script printed x: 10 and [x]: 4. The colleague says: “It works. Keep it and turn it into the parser.” What do you do?

Ask the agent to add tests to the script and grow it into the parser.Delete the script and tell the agent about [x] at the start of the next session.Move the script to a prototypes folder in the repository, in case it is useful later.Write the [x] rule into the spec, add an [x] case to the test, and delete the script.

DecideHintSkip

Small on purpose

In 1975, Fred Brooks advised teams to plan to throw away the first version of a whole system, because they would anyway. In the 1995 anniversary edition he took that advice back. He wrote that it assumed a system is designed, built and tested in one pass, and that he now preferred to grow a system step by step [3]. A throwaway prototype in this lesson is a different thing from a first version of the system. It is a few lines that answer one question that you wrote down before it existed. The system itself grows one checked component at a time, as Splitting the work into components the agent can build one at a time planned it.

Exercise

Take the spec you wrote in the exercise of Let the agent interview you, and pick one open question from it that a program can answer. Write the question in one sentence. If your spec has none, use the open question of sample/SPEC.md in your copy of that lesson’s fixture: whether archive list shows the date. A program that prints five archived items both ways shows you the two forms. The answer is then your decision, which you write into the spec. Start your coding agent in a scratch directory outside the fixture, so the prototype can’t touch the real code, and brief it for the smallest program that answers the question, with what to leave out and the one file it may create. Run the program, write the answer and your decision into the spec, and delete the program. Fifteen minutes is enough. You do this so that a design question is settled by what a program shows, and the code that showed it never reaches the system.

A good result: the question fits in one sentence and was written before the brief. The program is one file with no tests, and it is gone. The spec states the answer, and the question is no longer under the open questions. Reflect: how long would a discussion of the same question have taken, and would it have ended with the same answer?

Stretch: Take a second open question that only a person can decide, and write down why no program answers it.

Recap

1. A throwaway prototype is a small program built to learn, and it is thrown away once it has answered its question [1].
2. Write the question down in one sentence before you ask for code. A program that answers no named question grows into a first version of the system.
3. Brief the agent to leave out tests, error handling and structure, and to create one file. They are effort spent on code that you delete.
4. Write the answer and your decision into the spec, and add a case for it to the acceptance test, before the program is deleted. A fresh session reads only the spec.
5. Keep the prototype small. Brooks later took back his advice to throw away a whole first version [3], and the system grows one checked component at a time.

You can now

• Settles a design question with a throwaway prototype

Mark lesson finishedI know this, skip it

Habit

The next time you are about to argue about a design point with a colleague, ask whether a short program could settle it, and write the question down before anyone writes the program.

References

1. David Thomas, Andrew Hunt. The Pragmatic Programmer: Your Journey to Mastery, 20th Anniversary Edition. Addison-Wesley, 2019. Book. Thomas 2019
2. Bill Venners. Tracer Bullets and Prototypes: A Conversation with Andy Hunt and Dave Thomas, Part VIII. Artima Developer, 2003. Reference. Artima tracer bullets
3. Frederick P. Brooks, Jr.. The Mythical Man-Month: Essays on Software Engineering, Anniversary Edition. Addison-Wesley, 1995. Book. Brooks 1995