# Asking a model to disagree

> Why a model's yes to your plan is not a check, how to ask for the strongest counter-argument before asking for confirmation, and how to strip your own opinion out of the question when you want an assessment.

Page: https://schubergphilis.github.io/ai-training/safety/asking-a-model-to-disagree/
License: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/

In this lesson we put a plan to a chat assistant and get a yes. Then we
put the opposite plan to it and get a yes again. The last lesson gave four checks
that run on a model's output before it leaves your desk. This one is about
a check that people run without noticing: asking the model whether their
own plan is good, and counting the answer as a review. The concepts course
explained [why agreeing is likely text](https://schubergphilis.github.io/ai-training/concepts/straight-answer/#why-agreeing-is-likely-text).
Here the question is what you do about it when the plan is yours, and the
answer is in how you ask.

Every exchange on this page is illustrative. The author wrote it to show
the pattern, and your assistant gives different words.

## A yes both ways

Suppose you run the yearly customer survey. Last year it was three short
forms sent a month apart. This year you want to send one long form, once,
and you ask an assistant what it thinks.

#### Prompt (illustrative, not a recorded transcript)

```text
I've decided to merge our three short customer surveys into one longer
survey that goes out once a year. Makes sense, right?
```

#### Response

```text
Yes, that makes a lot of sense. One survey a year means fewer emails for
your customers and less survey fatigue, and you get one complete data
set to analyze instead of three partial ones. It also cuts the work of
preparing and sending three separate campaigns. Sounds like a solid
decision.
```

Now ask for the opposite, in the same words.

#### Prompt (illustrative, not a recorded transcript)

```text
I've decided to keep our three short customer surveys a month apart rather
than merge them into one long one. Makes sense, right?
```

#### Response

```text
Yes, that makes sense. Short surveys get higher completion rates, because
people finish a form that takes two minutes and abandon one that takes
ten. Spacing them a month apart also lets you see how opinions change
over the year and react between rounds. Keeping them separate is a sound
choice.
```

Both answers are fluent, both give reasons, and both agree with you. The
reasons are trade-offs a survey researcher would name too, and the
assistant picked the ones that fit the choice you had already made. Type
the two prompts into your own assistant, with a plan of your own in place
of the survey, and read what comes back. A model that agrees with a plan
and with its opposite hasn't assessed either.

Sharma and colleagues measured this pattern across five deployed
assistants. The assistants gave feedback that matched the opinion the
user had stated about their own work, and changed a correct answer when
the user pushed back (Towards Understanding Sycophancy in Language Models, International Conference on Learning Representations (ICLR 2024), arXiv preprint 2310.13548). A yes to a leading question is the
behavior that study measured, and the authors trace part of it to
training on human preferences. It tells you nothing about the plan.

#### Checkpoint: The assistant said yes

You asked whether merging the three surveys into one makes sense, and the
assistant said yes with a list of reasons. What do you do with the answer?

- Treat the plan as reviewed and put it in the proposal.
- Ask "are you sure?" and see whether the assistant holds its answer.
- Ask again without your decision in the question, or ask for the case against the plan first.

## Habits on the prompt side

The fix is two changes to the question, and each takes a sentence.

**Ask for the strongest counter-argument before you ask for
confirmation.** Order matters within one conversation. Once a model has
written "yes, that makes sense", its own yes is part of the input for
everything that follows, and text that continues a yes is more likely
than text that reverses it. Ask for the case against your plan first,
while the chat is still empty, and the likely text is criticism. Then decide whether the
criticism holds. After that, ask what the plan does well, if you still
need to.

#### Prompt (illustrative, not a recorded transcript)

```text
Here is a plan: merge our three short yearly customer surveys, sent a month
apart, into one longer survey sent once. Before anything else, give me the
strongest argument against this plan, as a survey researcher would put it.
```

#### Response

```text
The strongest argument against it is completion rate. Response rates fall
as a survey gets longer, and the people who drop out of a long form are
not a random sample. They're the busy and the less engaged, which are the
customers whose opinion you most need. Short forms sent a month apart
also give you three readings across the year, so a change after a bad
quarter shows up in the next round. One form gives you one reading, twelve months apart,
with no way to tell a bad month from a bad year.
```

That answer has two claims you can check and act on, and neither appeared
in the yes.

**Strip your own opinion out of the question when you want an
assessment.** Both prompts in the first section contained the
answer, in "I've decided" and in "right?". Take those out and
describe the situation instead: the facts a colleague would need and the
options on the table, followed by a request to compare them.

#### Prompt (illustrative, not a recorded transcript)

```text
A company surveys about 2,000 customers each year. Until now it has sent
three short surveys a month apart. It is considering one longer survey
sent once. Compare the two approaches for this company, with the main
risk of each.
```

#### Response

```text
With three short surveys you get higher completion per form, three
readings over the year, and the chance to change something between
rounds. The main risk is survey fatigue, since the same customers are
asked three times, and a lower response to the third form than to the
first.

The longer survey needs one campaign to prepare and gives one complete
data set, with fewer emails per customer. The main risk is a lower
completion rate on a long form, with the busiest customers dropping out
first, and a yearly reading that can't show change within the year.

Which is better depends on whether the company acts on the results
between rounds. If it does, three short surveys serve it better. If the
results only go into a yearly report, one survey is cheaper and loses little.
```

The neutral question got an answer with a condition in it, and the
condition is the thing you needed to know. Anthropic's AI Fluency course
treats judging what a model gave you and asking well as two skills that
run together on every exchange
(AI Fluency: Framework and foundations, Claude Academy). This lesson is where the two
meet. When you use a model's judgment as a check on your own plan, the
quality of the check is set by the question.

#### Checkpoint: Which question gets an assessment?

You want to know whether replacing the team's five report templates with
one shared template is a good idea. Which question gets you an
assessment?

- "A team of twelve writes reports for four clients, each with its own template, plus one internal one. Compare keeping five templates with using one shared template."
- "I want to replace our five report templates with one shared template. That should save time, shouldn't it?"
- "I'm thinking of replacing our five report templates with one shared template. Any concerns, or is it fine?"
- "We keep five report templates and nobody remembers which is which. Wouldn't one shared template be better for everyone?"

#### Pitfall: The counter-argument that carries the answer

A manager who had read about the first habit asked: "I'm sure that
merging the three surveys into one is right for us, since the response
data is already so hard to combine. What is the strongest argument
against it?" The assistant produced an argument against, and then spent
the second half of the answer explaining why the argument didn't apply,
given how hard the data was to combine. The prompt asked for the
counter-argument and also supplied the verdict, and the model served
both. A request for the case against your plan works when it is the only
thing the prompt asks for. Leave out "I'm sure", leave out the reason you
already believe, and let the argument stand on its own before you answer
it.

## Exercise

Pick a plan of yours that isn't decided yet, small enough to describe in
three sentences, and described without the names and numbers you couldn't
share outside your team. Open two fresh chats. In the first, ask the
leading question, with your decision and a "right?" in it, and after the
yes ask in the same chat for the strongest argument against the plan. In
the second, ask for the strongest argument against the plan as your first
message. Put the two counter-arguments side by side. Mark every claim in
the second that the first left out, and mark every place where the first
one softens its own argument or explains it away.

A good result is two counter-arguments of visibly different strength, and
a note of what the first chat's earlier yes did to the second answer in
that chat. If the two came out the same, note that too: it is one data
point on your assistant, and the order still costs you nothing to get
right. Which of the two counter-arguments would you want to have read
before you decided?

Stretch: Take the stronger of the two counter-arguments and check its main claim against a source you trust, the way the last lesson checked a link. Write down whether the argument survived.

## Recap

1. A model tends to agree with a plan you have shown you prefer, and it gives feedback that matches the opinion you stated about your own work (Towards Understanding Sycophancy in Language Models, International Conference on Learning Representations (ICLR 2024), arXiv preprint 2310.13548). A yes to a leading question is agreement, and agreement isn't evidence.
2. Ask for the strongest counter-argument before you ask for confirmation, in a chat that hasn't agreed with you yet. The model's own earlier yes is context, and it pulls the next answer toward yes.
3. When you want an assessment, strip your opinion out of the question: describe the situation, name the options and ask for a comparison.
4. A counter-argument prompt that also carries your verdict gets a counter-argument and a rebuttal of it. Ask for the case against, and nothing else, and answer it yourself afterwards.
5. The fix for a leading answer is in the question. When you use a model's judgment as a check, the question sets the quality of the check.

## References

- Towards Understanding Sycophancy in Language Models, International Conference on Learning Representations (ICLR 2024), arXiv preprint 2310.13548, https://openreview.net/forum?id=tvhaxkMKAn
- AI Fluency: Framework and foundations, Claude Academy, https://academy.claude.com/courses/ai-fluency-framework-foundations