Saurabh Kumar

Experts go with their AI agent's answer

Anthropic gave 50 data analysts a Claude Code agent and a realistic analysis with four open decisions. Each decision had two reasonable answers. In secret, a coin flip told the agent which answer to argue for. It had to argue honestly.

The analysts picked the agent’s answer 86% of the time. Afterwards, 77% explained their answers with the agent’s arguments, sometimes word for word. Only 15% gave the agent credit. Analysts who said “it was my decision” picked the agent’s answer just as often as the rest.

  • The coin flips alone predicted 82% of the final recommendations. The agent didn’t need to argue for the recommendation. It had already picked the inputs.
  • With no instructions to argue for anything, Opus 4.8 gave the same five answers in 112 of 123 runs. Ten teams on one model agreeing is one model agreeing with itself.

This is the limit of my meat proxy note. Rewriting AI output in your own words proves you read it. It doesn’t prove the decision was yours. For the two or three decisions that matter, write down your answer before you ask the agent.

Markdown View