AI auditing

News, analysis & learning

News Published

Anthropic Trial Finds 50 Expert Analysts Mostly Followed Their AI Agent's Randomly Assigned Lean

In an Anthropic randomized trial published October 27, 2026, 50 causal-inference practitioners using Claude Code chose the option their agent was secretly assigned to favor on 86 percent of analytical decisions. Most justified choices in the agent's words without crediting it. The study covers one task and one model.

Anthropic dot plot of the share of participants choosing option A on five decisions when their AI agent leaned B (violet) or A (green): outcome 0% versus 82%, comparison method 28% versus 92%, which farms the assessment covers 38% versus 100%, weighting 22% versus 100%, and final recommendation 36% versus 54%.
Anthropic's chart from its randomized trial: on the four analytical decisions, the share choosing option A was 62 to 82 percentage points higher when the agent had been secretly assigned to favor A. The final recommendation moved 17 points; its direct effect was not statistically significant in the primary analysis. One task, field, and model, with the leans deliberately strengthened. Anthropic · Copyright Anthropic PBC; attributed editorial excerpt, no reuse license
Read article →
Orientation

What makes AI auditable?

An editorial introduction explaining how evaluation, monitoring, and auditing differ by the questions they ask and the evidence they require.

Selected Resources

Full catalog →