Wrong about the world, or wrong about the text

The lesson on how models work named hallucination: fluent, confident text with nothing behind it. A wrong answer can be wrong in two different ways, though. It can be wrong about the world, such as a date that never happened. It can also be wrong about the text you gave the model, such as a summary that changes a date in the memo you pasted. In this lesson we split one wrong summary into those two kinds and find the check that catches each one.

One summary, five mistakes

You paste a short memo into a chat assistant and ask for a summary. The exchange below is illustrative: the author wrote it to show the pattern, and a real model makes different mistakes.

Prompt (illustrative, not a recorded transcript)

Summarize this memo in two bullet points, and say which country the offsite is in.

Memo: The team offsite moves from 12 March to 19 March. It is held in our Lisbon office. The budget is capped at 4,000 euros. The office manager books the rooms.

Response

- The offsite moves from 12 March to 26 March, one week later
  than planned, and takes place in the Lisbon office.
- The budget is capped at 4,000 euros.
- The office manager books the rooms and the flights.

The offsite is in Spain, since Lisbon is the capital of Spain.

The reply reads well, and it gets five things wrong.

• It gives 26 March, and the memo says 19 March.
• It says the move is one week, and 12 to 26 March is two weeks. The reply contradicts itself.
• It has three bullet points, and you asked for two.
• It says the office manager books the flights, which the memo never mentions.
• It puts Lisbon in Spain. Lisbon is the capital of Portugal.

The first four mistakes you can find with the memo alone. The fifth one you can’t, because the memo never says which country Lisbon is in.

Two kinds of hallucination

A survey of research on hallucination in large language models sorts these mistakes into two groups, and the names of the groups are factuality and faithfulness [1].

A factuality error is wrong about the world. The survey separates a claim about something real that gets it wrong, such as the wrong person named as the inventor of a device, or Lisbon placed in Spain, from a claim that can’t be checked against anything that exists, such as a quote from a person who never said it or a reference to a paper nobody wrote [1].

A faithfulness error departs from what the user gave the model, or from itself. The survey divides it by what the output departs from [1]:

• The output ignores the instruction, such as three bullet points where you asked for two.
• The output changes or adds to the text it was given, such as 26 March where the memo says 19 March, or flights that the memo doesn’t mention.
• The output contradicts itself, such as “one week later” next to two dates that are two weeks apart.

So the summary above has one factuality error and four faithfulness errors. A faithfulness error can sound true. “The office manager books the flights” is a reasonable thing for an office manager to do, and it can even be true in your company. It is still a mistake in this summary. A summary reports what the memo says, and the memo doesn’t say it.

Checkpoint · sort

Wrong about the world, or about the input?

Unplaced (drag an item to a bucket, or select it and then click the bucket)

Given a memo that says the meeting is on Tuesday, the reply says Wednesday.Asked with no document for the boiling point of water at sea level, the reply says 90 °C.Asked for one sentence, the reply is five sentences long.Asked with no document for research on remote work, the reply cites a study that was never published.The reply says the agenda has two items, then lists three.

Wrong about the world (factuality)

Departs from the input or from itself (faithfulness)

CheckHintSkip

Which check finds each kind

The two kinds need different checks, and this is the reason to tell them apart.

To find a factuality error, you check the claim against a source outside the chat, such as a reference work or the official page. The survey describes detection methods that split an answer into single facts and look each one up in a trusted source of knowledge [1]. Asking the same model isn’t that check. The survey notes that a model isn’t a reliable store of facts [1].

To find a faithfulness error, you compare the output with the input: the instruction you wrote and the text you pasted. The survey describes detection methods that measure how many of the facts in the output also appear in the source content [1]. For you, that means reading the memo and the summary side by side. This check doesn’t need outside knowledge, and outside knowledge doesn’t help. A web search can tell you a lot about Lisbon, and nothing about which date this memo gave. The lesson on spotting a hallucination runs this check on a summary of a council report.

The two checks don’t replace each other. A summary can be faithful to a memo that is itself wrong. An outside source is the check that catches that. An answer can be true about the world and still unfaithful, like the flights that the office manager may well book.

The survey groups the ways to reduce hallucination by their cause in the training data, the training and the way the model generates text [1], so it doesn’t give one remedy for each kind. What you can do as a user is know which check a claim needs.

Checkpoint · choice

Which check finds it?

You paste a contract into a chat assistant and ask for the notice period. It answers “three months”. Which check tells you whether that answer is right for this contract?

Search the web for the usual notice period in contracts of this kind.Ask the same assistant whether it is sure, in a new chat.Find the clause on notice in the contract and read it.Ask a second assistant, without the contract, what notice periods usually are.

CheckHintSkip

Pitfall: Checking a summary against the web

You ask an assistant for a short version of a supplier’s letter, and it says the price goes up by 5 percent from January. That sounds plausible. A web search shows that prices in that market rose about 5 percent this year, so you send the summary on. The letter said 8 percent from April. The search answered a question about the world, and the mistake was about the letter. Rule: when the model worked from a text you gave it, check its output against that text first.

Exercise

Copy the wrong answers below, all from chat assistants, into a note, and for each one write two things: the kind of error (factuality or faithfulness) and the check that finds it, in one line, such as “compare with the menu”. The point is to practice choosing the check before you start checking, because the wrong check can pass a wrong answer.

1. Asked to translate a menu, the reply adds a dish the menu doesn't have.
2. Asked who wrote a well-known novel, the reply names the wrong author.
3. Asked for three options, the reply gives five.
4. Asked about a city's population, the reply quotes a census that was
   never held.
5. Asked to summarize a letter, the reply says the letter was signed by
   the manager, and later in the same reply that nobody signed it.

It takes about ten minutes. A good result puts 2 and 4 under factuality and 1, 3 and 5 under faithfulness. Each check line for factuality names an outside source you would open, and each line for faithfulness names the part of the input you would compare with. Reflection: for answer 5, do you need the letter to see the mistake, or the reply alone?

Stretch: Paste a short email you wrote, one with nothing confidential in it, into a chat assistant, ask for a two-line summary and one fact about a place or a date the email names, and sort every mistake you find in the reply the same way.

Recap

1. A factuality error is wrong about the world: a real thing described wrongly, or a claim nothing can confirm, such as a quote nobody said [1].
2. A faithfulness error departs from what you gave the model, or from itself: it ignores the instruction, changes or adds to the text you pasted, or contradicts itself [1].
3. Check a factuality claim against a source outside the chat. Check a faithfulness claim against your own input, side by side.
4. A faithful answer can still be wrong about the world, and a true statement can still be unfaithful, so a claim that matters may need both checks.

You can now

• Names the common ways output goes wrong and why

Mark lesson finishedI know this, skip it

References

1. Lei Huang, Weijiang Yu, Weitao Ma and 8 others. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Transactions on Information Systems 43(2), Article 42, 2025, arXiv preprint 2311.05232. Paper. Huang 2025