Anthropic urges ‘temporary pause’ on AI development to discuss risks | AI (artificial intelligence)
At its last meeting, Anthropic floated the idea of a worldwide “temporary pause” on AI development and said it would bring together “policymakers” to discuss the dangers of advanced AI. release highlights the capabilities of its products.
In a lengthy post published Thursday, Anthropic detailed its AI model Claude’s progress toward “recursive self-improvement,” meaning he can make better, stronger versions of himself. Recursive self-improvement is the bogey of AI security researchers, seen as the key step in AI becoming superintelligent and thus leading to widespread consequences for humanity.
This idea features heavily in the widely read AI 2027 doomsday scenario Last year’s movie imagines AI agents designing increasingly intelligent versions of themselves, one of which eventually kills all of humanity with a bioweapon to make room for more data centers and solar panels.
Anthropic’s post notes a “trend” of increasing capability in Claude that “points to an AI system that, when pushed far enough and given enough computing… can design and develop its own successor completely autonomously.” This could increase the risk of “humans losing control over AI systems,” Anthropic said.
To deal with this, Anthropic proposed holding conversations where “policymakers, researchers, civil society, and other AI companies can help answer some of the questions this paper raises.”
The news comes alongside a separate report from the Financial Times that the US AI company has placed engineers. in The National Security Agency, despite a legal battle with the Pentagon over the use of its tools. Engineers reportedly helped the NSA use Anthropic’s model Mythos for offensive cybersecurity operations.
Steven Murdoch, a professor at University College London, said that if calling for a worldwide debate on AI risk was at odds with supporting a US spy agency likely to attack Iran and China with cyber weapons, neither development was “surprising” given the AI company’s past actions.
“Anthropic may give the impression of being warm and fuzzy, but their definition of AI security is narrow. Supporting U.S. officials in the development of offensive capabilities has never been something they have opposed,” he said.
Murdoch said Anthropic’s post did not provide evidence of any step change in the advancement of AI capabilities.
“It is true that there is some evidence that AI capabilities have increased and continue to increase with no boundaries immediately clear,” he said, but added: “I don’t think anything has fundamentally changed today that led Anthropic to publish this paper.”
The advances that Anthropic details in its post do not mean that artificial intelligence improves itself over and over again; at least for now. Instead, the company reports that a significant part of the work done to improve artificial intelligence systems is now done with artificial intelligence. Claude is said to be good at “running experiments” or at least speeding up certain parts of the code.
Like other AI systems, Claude appears to be improving at tackling more challenging tasks. Anthropic described this as “driving research” and “proposing its own experiments”, but these successes appear to have occurred within strict boundaries and were limited to coding-related tasks.
Anthropic said the quality of code written by the AI has also improved: “As of May 2026, over 80% of the code we have assembled in Anthropic’s codebase has been written by Claude.”
Murdoch says Anthropic calls for ‘temporary pause’ on AI echoed There have been other suggestions regarding AI security over the years, such as the company’s plan to engage with policymakers. “It’s a reminder of what they’re worried about and have been worried about for years.”
“I’m sure the attention has been welcomed, but again, this is nothing new. Antropik has been trying to attract the attention of policymakers since its inception.”
Two months ago, Anthropic announced Mythos, an artificial intelligence model that it implied was too powerful to be released publicly due to cybersecurity concerns — but refused to release it. The announcement caused widespread uproar and attracted the attention of the US treasury secretaryScott Bessent and MI5.
But some experts have suggested that there may be more hyperbole than substance behind Anthropic’s Mythos announcement, especially given the vagueness with which the company describes some of Mythos’ capabilities. Heidy Khlaaf, chief artificial intelligence scientist at the AI Now Institute, called Mythos’ announcement “a marketing piece.”
Anthropic this week filed for an IPO valuing the company at $1 trillion. The company has been contacted for comment.
