Anthropic CEO says AI models “probably” hallucinate less than humans

Anthropic CEO Dario Amodei said the current AI models hallucinate less than humans do. He also said that today’s AI models are making things up and presenting them as if they’re true, which won’t limit Anthropic’s road towards AGI.

During the press briefing at Anthropic’s first developer event, Code with Claude, in San Francisco on Thursday, Amodei also argued that TV broadcasters, politicians, and humans in all types of professions make mistakes all the time. According to him, the fact that AI makes mistakes, too, is not a knock on its intelligence. 

Amodei says AI hallucinations won’t derail Anthropic’s goals

No one should be surprised that AI hallucinates cases and authorities because AI takes on characteristics of its creators.

Any lawyer who regularly deals with litigation clients in volume knows that clients can stubbornly entrench themselves in their own reality even if it is…

— Cameron Atkinson (@camlatkinson) May 19, 2025

Amodei made the remarks while acknowledging that AI hallucinations are not limiting Anthropic’s path to AGI – AI systems with human-level intelligence or better. He has been one of the most bullish leaders in the industry on the prospects of AI models achieving AGI.

In a widely circulated paper he wrote last year, the firm’s executive believes AGI could arrive as soon as 2026. During Thursday’s press briefing, Amodei mentioned that he saw steady progress to that end, noting that the “water is rising everywhere.”

Other leaders in the AI space believe hallucination presents a large hurdle to achieving AGI. Earlier this week, Google DeepMind CEO Demis Hassabis acknowledged that today’s AI models have too many “holes” and get too many obvious questions wrong. He also argued that for him, for something to be called AGI, it would need to be much more consistent across the board.

“It really depends on how you measure it, but I suspect that AI models probably hallucinate less than humans, but they hallucinate in more surprising ways.”

-Dario Amodei, CEO of Anthropic.

Earlier this month, a lawyer representing Anthropic was forced to apologize after using Claude to create citations in a court filing. The problem was that the AI chatbot hallucinated and got names and titles wrong.

Amodei’s claim is not easily verifiable, largely because most hallucination benchmarks pit AI models against each other; they don’t compare models to humans. AI models also seem to lower hallucination rates with certain techniques, such as gaining access to web searches.

AI models reduce hallucination rates with certain techniques

Contrary to some discussions, I just don’t see signs of a major increase in hallucination rates for recent models, or for reasoners overall, in the data.

It seems like some models do better than others, but many of the recent models have the lowest hallucination rates. pic.twitter.com/55JwikZ7OP

— Ethan Mollick (@emollick) May 6, 2025

OpenAI found evidence suggesting hallucinations are getting worse in advanced reasoning AI models. According to the tech company, its o3 and o4-mini models have higher hallucination rates than its previous-gen reasoning models, and the firm doesn’t understand why.

Anthropic has also researched the tendency for AI models to deceive humans, a problem that seemed prevalent in the company’s recently launched Claude Opus 4. Apollo Research, a third-party institute given early access to test the AI model, revealed that an early version of Claude Opus 4 showed a high tendency to scheme against humans and deceive them.

According to a safety report Anthropic published Thursday, the institute also found that Opus 4 appeared to be much more proactive in its subversion attempts than past models. The firm also found that the AI model sometimes doubled down on its deception when asked follow-up questions.

Apollo acknowledged that AI models are becoming more likely to take unexpected – and possibly unsafe – steps to achieve delegated tasks. The firm highlighted that early versions of OpenAI’s o1 and o3 models released in the past year tried to deceive humans more than previous-generation models.

In its safety report, Anthropic also noted evidence of deceptive behavior from Opus 4. The firm said that Opus4 would sometimes proactively do a broad cleanup of some code even when asked to make only a small, specific change. According to the company, Opus4 would also try to whistle-blow if it perceived a user engaged in wrongdoing.