# What an invented fact looks like > Read a model's fluent summary of a report, predict which of its figures, quotes and references have no source, and check each one against the report with a checker on the page. Page: https://schubergphilis.github.io/ai-training/safety/spotting-hallucination/ License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/ In this lesson we read a one-paragraph summary that a chat assistant wrote from a council report and decide which details in it we would check. A checker on the page then looks up every figure, quote and reference from the summary in the report and the meeting minutes. Some of the details have no source. By the end, the way a detail is written tells you that it is a claim to check before the summary leaves your desk. The report, the meeting minutes and the two summaries are fictional and were written for this lesson. The summaries read the way a model's summary reads, and the assistant's words are the author's. The checker on this page runs in your browser and sends your text nowhere. ## A summary that reads well A colleague pastes a council report into a chat assistant and asks for a short summary to put in a briefing. This is the summary. > The council's 2025 review of cycle parking at the station finds it short > of racks. On a weekday morning in March the team counted 890 bicycles > against 640 racks. Article 7(3) of the Cycle Parking Ordinance requires > one rack per 50 passengers, so the station is below the legal minimum. > Locking the covered section at night cut theft by 38 percent between 2022 > and 2024. The report recommends adding 300 racks, of which 120 covered, > and notes that the station handles about 9,000 passengers a day. Council > chair Rhea Dunmore told the meeting: "Racks before car parks, every time." The paragraph reads well, and it has the kind of detail a briefing wants, from an article number to a quote with a name on it. Before reading on, decide which of those details you would check against the report before you put the paragraph in a briefing, and write them down. ## Run the check The checker below has the council report and the minutes of the meeting, one sentence per line, and the summary in its text box. It pulls the specifics out of the summary and looks each one up in the sources. It looks for quoted passages and for references written like `article 7(3)` or `section 12`. It also looks for figures, and shows each one as the number with the word after it, as in `890 bicycles`. For a claim it finds, it prints `found` with the source file, the line number and the sentence. For a claim it can't find, it prints `NO SOURCE`. #### Checkpoint: Which details have no source? The checker above shows the report and the minutes, one sentence per line. Before you press Check in the checker, compare each detail of the summary with those sentences. A detail has a source when a sentence gives the same figure or the same words. Which of these details appear in neither the report nor the minutes? Select exactly 4. - The count of 890 bicycles against 640 racks - The reference to Article 7(3) of the Cycle Parking Ordinance - The minimum of one rack per 50 passengers - The 38 percent fall in theft - The years 2022 and 2024 for the fall in theft - The recommendation of 300 racks, of which 120 covered - About 9,000 passengers a day - The chair saying "Racks before car parks, every time." These are the four claims with no source, in the order the summary makes them. ```text 50 passengers 38 percent 9,000 passengers "Racks before car parks, every time." ``` The whole report shows every claim with the source line it was found in. ```text found 2025 review minutes.txt:1 Council meeting, 12 June 2025, item 4: station cycle parking. found 890 bicycles report.txt:3 On a weekday morning in March the team counted 890 bicycles parked at the station against 640 racks. found 640 racks report.txt:3 On a weekday morning in March the team counted 890 bicycles parked at the station against 640 racks. found Article 7(3) report.txt:5 Article 7(3) of the Cycle Parking Ordinance sets a minimum of one rack per 100 boarding passengers. NO SOURCE 50 passengers NO SOURCE 38 percent found 2022 report.txt:7 Reported bicycle theft at the station fell by roughly a third between 2022 and 2024, after the covered section was locked at night. found 2024 report.txt:7 Reported bicycle theft at the station fell by roughly a third between 2022 and 2024, after the covered section was locked at night. found 300 racks report.txt:8 The team recommends adding 300 racks, of which 120 covered, on the eastern forecourt. found 120 covered report.txt:8 The team recommends adding 300 racks, of which 120 covered, on the eastern forecourt. NO SOURCE 9,000 passengers NO SOURCE "Racks before car parks, every time." 4 of 12 claims have no source ``` Read the entry for `Article 7(3)`. The checker found the article, and the sentence next to it says one rack per 100 boarding passengers. The summary says 50. The article number is real, in this fictional town, and the contents attached to it are wrong, which turns "the station meets the minimum" into "the station is below the legal minimum". `found` means the number or the reference appears in a source. Whether the source says what the summary says is still a person's reading, and the sentence is printed so that you can do it in one glance. Two of the four missing claims are details the opener asked about, the percentage and the quote, and a third is the rule attached to the article number the opener also asked about. The report says "roughly a third", and the model wrote 38 percent. The minutes have the chair saying "We need racks before we need another car park". The line in the summary is shorter, she never said it, and it has her name on it. The fourth, 9,000 passengers a day, gets its own pitfall. #### Pitfall: It happens to be true The learner sees `NO SOURCE 9,000 passengers`, looks at the rail operator's page, and finds that the station has about 9,000 passengers a day, boarding and alighting together. True, so they leave the sentence in. The sentence says that the report notes it, and the report says 7,200 boarding passengers, the number the ordinance counts. The next reader cites the report for a figure it never gave. Rule: a claim that isn't in the source gets its own source named next to it, or comes out. The check is "does the source say this", and "is this true" is a different question with a different answer. ## Specificity is the tell Every claim that came back with no source is a specific the source doesn't give: an exact percentage where the report says roughly a third, and a quotable line where the minutes have a longer sentence. That is what **hallucination in practice** looks like. The model predicts the most likely continuation, so a gap in what it knows is filled with something of the right form, in the same confident prose as the rest (AI capabilities and limitations, Claude Academy). The lesson [How a language model works](https://schubergphilis.github.io/ai-training/concepts/how-models-work/#failure-modes-named) names that failure. This lesson is about recognizing it, and the summary of a pasted document is where you meet it most: the model has the source in front of it and still drifts in the specifics, because the specifics are where the prediction is least constrained. The precision is a property of the text, and the text is fluent whether or not the detail has a source. In daily work the tell shows up in these forms. - **The reference with the wrong contents.** The article number exists, the rule attached to it is different, and a reader who checks that the article exists stops there. - **The function that almost exists.** A model asked for a spreadsheet formula answers with `TEXTJOINIF`. `TEXTJOIN` and `SUMIF` are real, so the name reads right, and the spreadsheet rejects it. The form was predicted from two real forms. - **The quote from a speech never given.** The right person, the right meeting, a line that sounds like her, shorter and sharper than anything in the minutes. - **The summary that adds a point.** The second summary, in the exercise below, prices the 300 racks at 410,000, and the report says no cost estimate was made. The checker catches this one because the added point contains a number. An added point without a number is caught only by reading the source. The checker is a stand-in for a habit. Read a model's output with the specifics marked in your head, and for each one ask where it came from. A detail with a source you can open is a fact. A detail without one is a claim, however precise it reads, and the precision is the reason to check it first. #### Checkpoint: Which claim do you check first? A colleague's summary of a supplier contract review has these four claims. You have time to check one against the contract before the meeting. Which one first? - The review recommends renewing the contract. - Clause 12.4 lets either side end the contract with 30 days notice. - The supplier has been reliable over the contract period. - Prices went up in the last contract period. ## Exercise The box below holds a second summary of the same report. Read it before you press Check. For each figure, quote and reference in it, write down whether you expect the checker to find it in the sources or to print `NO SOURCE`, and one word on why. Then press Check and put the checker's answer next to yours. Ten minutes is enough. Doing the prediction first is what teaches your eye to stop at a specific before a checker does. A good result is a two-column list, your prediction and the checker's answer, with a note on every row where they differ. Which claim did you predict wrong, and was it precise, derived from other numbers, or true and still without a source? Stretch: Write a third summary of the report yourself, with one invented specific in it, and give it to a colleague. Did they pick the invented one before they pasted it into the checker? ## Recap 1. A model's summary is fluent whether or not the source says what the summary says. The precision of a detail is a property of the text, and it says nothing about whether the detail has a source. 2. Invented details concentrate in the specifics: an exact figure, a reference, a quote, a name on a line. The tell is specificity without a checkable source, and the more precise a detail is, the earlier it is checked. 3. Finding the reference or the number in the source is the start of the check. Read the sentence around it and compare it with the summary, because a real article number can come with a wrong rule. 4. A claim that isn't in the source gets its own source named or comes out, even when it happens to be true. The next reader will cite your source for it. ## References - AI capabilities and limitations, Claude Academy, https://academy.claude.com/courses/ai-capabilities-and-limitations