# Wrong about the world, or wrong about the text

> Two kinds of wrong answer, one wrong about the world and one wrong about the text you gave the model, and the check that finds each kind.

Page: https://schubergphilis.github.io/ai-training/concepts/wrong-about-the-world/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

The lesson on how models work named
[hallucination](https://schubergphilis.github.io/ai-training/concepts/how-models-work/#failure-modes-named): fluent,
confident text with nothing behind it. A wrong answer can be wrong in two
different ways, though. It can be wrong about the world, such as a date
that never happened. It can also be wrong about the text you gave the
model, such as a summary that changes a date in the memo you pasted. In
this lesson we split one wrong summary into those two kinds and find the
check that catches each one.

## One summary, five mistakes

You paste a short memo into a chat assistant and ask for a summary. The
exchange below is illustrative: the author wrote it to show the pattern,
and a real model makes different mistakes.

#### Prompt (illustrative, not a recorded transcript)

```text
Summarize this memo in two bullet points, and say which country the
offsite is in.

Memo: The team offsite moves from 12 March to 19 March. It is held in
our Lisbon office. The budget is capped at 4,000 euros. The office manager
books the rooms.
```

#### Response

````text
```text
- The offsite moves from 12 March to 26 March, one week later
  than planned, and takes place in the Lisbon office.
- The budget is capped at 4,000 euros.
- The office manager books the rooms and the flights.

The offsite is in Spain, since Lisbon is the capital of Spain.
```
````

The reply reads well, and it gets five things wrong.

- It gives 26 March, and the memo says 19 March.
- It says the move is one week, and 12 to 26 March is two weeks. The reply
  contradicts itself.
- It has three bullet points, and you asked for two.
- It says the office manager books the flights, which the memo never
  mentions.
- It puts Lisbon in Spain. Lisbon is the capital of Portugal.

The first four mistakes you can find with the memo alone. The fifth
one you can't, because the memo never says which country Lisbon is in.

## Two kinds of hallucination

A survey of research on hallucination in large language models sorts
these mistakes into two groups, and the names of the groups are
factuality and faithfulness (A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, ACM Transactions on Information Systems 43(2), Article 42, 2025, arXiv preprint 2311.05232).

A **factuality** error is wrong about the world. The survey separates a
claim about something real that gets it wrong, such as the wrong person
named as the inventor of a device, or Lisbon placed in Spain, from a
claim that can't be checked against anything that exists, such as a
quote from a person who never said it or a reference to a paper nobody
wrote (A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, ACM Transactions on Information Systems 43(2), Article 42, 2025, arXiv preprint 2311.05232).

A **faithfulness** error departs from what the user gave the model, or
from itself. The survey divides it by what the output departs from
(A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, ACM Transactions on Information Systems 43(2), Article 42, 2025, arXiv preprint 2311.05232):

- The output ignores the instruction, such as three bullet points where
  you asked for two.
- The output changes or adds to the text it was given, such as 26 March
  where the memo says 19 March, or flights that the memo doesn't mention.
- The output contradicts itself, such as "one week later" next to two
  dates that are two weeks apart.

So the summary above has one factuality error and four faithfulness
errors. A faithfulness error can sound true. "The office manager books
the flights" is a reasonable thing for an office manager to do, and it
can even be true in your company. It is still a mistake in this
summary. A summary reports what the memo says, and the memo doesn't say
it.

#### Checkpoint: Wrong about the world, or about the input?

Buckets:

- Wrong about the world (factuality)
- Departs from the input or from itself (faithfulness)

Items:

- Given a memo that says the meeting is on Tuesday, the reply says Wednesday.
- Asked with no document for the boiling point of water at sea level, the reply says 90 °C.
- Asked for one sentence, the reply is five sentences long.
- Asked with no document for research on remote work, the reply cites a study that was never published.
- The reply says the agenda has two items, then lists three.

## Which check finds each kind

The two kinds need different checks, and this is the reason to tell them
apart.

To find a **factuality** error, you check the claim against a source
outside the chat, such as a reference work or the official page.
The survey describes detection methods that split an answer into single
facts and look each one up in a trusted source of knowledge (A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, ACM Transactions on Information Systems 43(2), Article 42, 2025, arXiv preprint 2311.05232).
Asking the same model isn't that check. The survey notes that a model
isn't a reliable store of facts (A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, ACM Transactions on Information Systems 43(2), Article 42, 2025, arXiv preprint 2311.05232).

To find a **faithfulness** error, you compare the output with the input:
the instruction you wrote and the text you pasted. The survey describes
detection methods that measure how many of the facts in the output also
appear in the source content (A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, ACM Transactions on Information Systems 43(2), Article 42, 2025, arXiv preprint 2311.05232). For you, that means reading
the memo and the summary side by side. This check doesn't need outside
knowledge, and outside knowledge doesn't help. A web search can tell you a
lot about Lisbon, and nothing about which date this memo gave. The lesson on
[spotting a hallucination](https://schubergphilis.github.io/ai-training/safety/spotting-hallucination/) runs this
check on a summary of a council report.

The two checks don't replace each other. A summary can be faithful to a
memo that is itself wrong. An outside source is the check that catches
that. An answer can be true about the world and still unfaithful, like
the flights that the office manager may well book.

The survey groups the ways to reduce hallucination by their cause in
the training data, the training and the way the model generates text
(A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, ACM Transactions on Information Systems 43(2), Article 42, 2025, arXiv preprint 2311.05232), so it doesn't give one remedy for each kind. What you can
do as a user is know which check a claim needs.

#### Checkpoint: Which check finds it?

You paste a contract into a chat assistant and ask for the notice period.
It answers "three months". Which check tells you whether that answer is
right for this contract?

- Search the web for the usual notice period in contracts of this kind.
- Ask the same assistant whether it is sure, in a new chat.
- Find the clause on notice in the contract and read it.
- Ask a second assistant, without the contract, what notice periods usually are.

#### Pitfall: Checking a summary against the web

You ask an assistant for a short version of a supplier's letter, and it
says the price goes up by 5 percent from January. That sounds plausible.
A web search shows that prices in that market rose about 5 percent this
year, so you send the summary on. The letter said 8 percent
from April. The search answered a question about the world, and the
mistake was about the letter. Rule: when the model worked from a text
you gave it, check its output against that text first.

## Exercise

Copy the wrong answers below, all from chat assistants, into a note,
and for each one write two things: the kind of error (factuality or
faithfulness) and the check that finds it, in one line, such as "compare
with the menu". The point is to practice choosing the check before you
start checking, because the wrong check can pass a wrong answer.

```text
1. Asked to translate a menu, the reply adds a dish the menu doesn't have.
2. Asked who wrote a well-known novel, the reply names the wrong author.
3. Asked for three options, the reply gives five.
4. Asked about a city's population, the reply quotes a census that was
   never held.
5. Asked to summarize a letter, the reply says the letter was signed by
   the manager, and later in the same reply that nobody signed it.
```

It takes about ten minutes. A good result puts 2 and 4 under factuality
and 1, 3 and 5 under faithfulness. Each check line for factuality names
an outside source you would open, and each line for faithfulness names
the part of the input you would compare with. Reflection: for answer 5,
do you need the letter to see the mistake, or the reply alone?

Stretch: Paste a short email you wrote, one with nothing confidential in it, into a chat assistant, ask for a two-line summary and one fact about a place or a date the email names, and sort every mistake you find in the reply the same way.

## Recap

1. A factuality error is wrong about the world: a real thing described
   wrongly, or a claim nothing can confirm, such as a quote nobody said
   (A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, ACM Transactions on Information Systems 43(2), Article 42, 2025, arXiv preprint 2311.05232).
2. A faithfulness error departs from what you gave the model, or from
   itself: it ignores the instruction, changes or adds to the text you
   pasted, or contradicts itself (A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, ACM Transactions on Information Systems 43(2), Article 42, 2025, arXiv preprint 2311.05232).
3. Check a factuality claim against a source outside the chat. Check a
   faithfulness claim against your own input, side by side.
4. A faithful answer can still be wrong about the world, and a true
   statement can still be unfaithful, so a claim that matters may need
   both checks.

## References

- A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, ACM Transactions on Information Systems 43(2), Article 42, 2025, arXiv preprint 2311.05232, https://doi.org/10.1145/3703155