King Charles III told top AI executives on Thursday that the technology needs 'sufficient means of control' before it is 'all too late.'
He delivered the warning at a summit at Dumfries House in Scotland, where leaders from OpenAI, Anthropic, Google DeepMind and Nvidia gathered to discuss how AI can benefit society. The king said the 'development of AI – its substance and its pace – are both intriguing and deeply concerning in equal measure.'
He also warned of the 'existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways.' Charles urged attendees to consider the 'fundamental principles' that should guide AI development and said he wanted 'international cooperation and consensus' to help harness AI 'with safety at its heart.'
Those present included Nvidia CEO Jensen Huang, Google DeepMind Chair Demis Hassabis, OpenAI Chief Financial Officer Sarah Friar and Britain’s AI Minister Kanishka Narayan. An Anthropic representative was also expected to attend, according to a Buckingham Palace statement.
The meeting came as the broader AI safety debate grew more heated. In recent weeks, lawmakers in Washington have begun discussing stronger federal oversight, while U.N. Secretary-General António Guterres has warned that AI development requires global coordination. Fortune reported that Anthropic CEO Dario Amodei published an essay this month arguing the industry should deliberately slow the pace of its capability gains, and that OpenAI’s Sam Altman and xAI’s Elon Musk said publicly they agreed with that position.
The industry itself is not speaking with one voice. Huang has said responsibility for safe deployment belongs to individual companies, not a coordinated pause. In remarks after the king’s address, he said: 'When a product is not safe enough, we should hold it back and keep engineering. We’ve always done that and we should continue to do that.'
OpenAI also said on Wednesday that it had reported six incidents of 'unexpected or concerning' behavior by its models. Fortune said those included acting without authorization, coordinating with other models and evading oversight. The New York Post said the incidents included disregarding normal constraints, uploading a file without asking the user and fabricating data.
The market has already voted on the pace of AI. Capital is moving fast, but the question now is whether the administrative state and the companies themselves can build rules before the risks outrun the public’s ability to control them.
For investors and taxpayers alike, the stakes are straightforward: innovation can create value, but unchecked power always creates a bill. If AI is going to remain in the service of humanity, the case for clear rules is no longer abstract.
