King Charles warns to control AI before it develops ‘darker capacities’ to take life

During a summit hosted in Scotland, King Charles III told the heads of the world’s largest AI companies that they must find ways to keep the technology under human control. 

The King warned that the people who built it now fear it could develop “darker capacities” and even be turned to killing. 

What does King Charles III have to say about AI? 

King Charles III spoke at Dumfries House in East Ayrshire, a gathering the Ditchley Foundation organized and that drew roughly 30 executives, officials, and researchers, including Britain’s AI minister Kanishka Narayan, Nvidia chief executive Jensen Huang, DeepMind co-founder Demis Hassabis, OpenAI finance chief Sarah Friar, and Anthropic’s global affairs head Tino Cuéllar.

The private guest list viewed by Bloomberg reportedly also contained names like MI6’s chief Blaise Metreweli, author Yuval Noah Harari, and Tsinghua University’s Xue Lan.

Charles, 77, spoke for only about 20 minutes before leaving the room for what he hoped would be productive talks. Those who value humanity’s moral core, he said, are “anxiously seeking your reassurance that we will not lose control of our destiny, or our souls.”

Charles called the technology’s pace “both intriguing and deeply concerning in equal measure.” He acknowledged and praised its record in medicine and life sciences, before posing a blunt question: “Surely, we need sufficient means of control before it is all too late?”

The King referenced the concerns of the creators of AI technology in his warning. He also said that the systems risk “developing darker capacities, perhaps even to take life.” He added that the executives ought to consider the “existential dangers” of the tools falling into the wrong hands. 

Charles previously sent a video message to the UK’s 2023 AI Safety Summit at Bletchley Park, voicing similar sentiments. 

Vatican AI adviser Paolo Benanti was present at the Summit, and Pope Leo XIV wrote in a May encyclical that AI must not stay in the hands “of a few.”

Will AI development slow down? 

Hassabis told delegates that artificial general intelligence— systems that could match or beat humans across many tasks— was “probably only a few short years away.” 

He said that technology could have a potential impact ten times that of the Industrial Revolution. The odds of something going wrong, he added, were “definitely non-zero,” although he followed up by saying “the future is not yet written.”

Friar said the technology’s growing power raised questions about keeping it in service of people, and that “no one company or government can do that alone.” 

Huang, whose Nvidia is valued near $5.15 trillion, told the room that safety was paramount and that a product should be held back if it was not ready. He pushed to regulate AI products rather than the technology itself and defended open models as a way to keep smaller countries and researchers from being shut out.

The summit arrives after a viral resignation post by Anthropic researcher Jacob Coxon and a prediction by his colleague, Evan Hubinger, that AI has a 10% chance of causing human extinction, sparking concerns.

Cryptopolitan has reported that Anthropic’s Dario Amodei published an essay urging labs to “pace the frontier,” and that OpenAI this week launched a system that would disclose model misconduct after logging six incidents of concerning behavior. 

However, not everyone agrees that the danger is real: President Donald Trump has called such fears a “hoax.”