The numbers come first. Anthropic said its model Claude is now leading 26% of its model research and development, helping the company develop the next, more intelligent version of itself. OpenAI, meanwhile, said this month that it has developed an automated 'research intern' and is moving toward an automated AI 'researcher' by March 2028.
The debate is no longer about distant theory. Developers say artificial intelligence is approaching 'recursive self-improvement,' a process in which models improve themselves and build their successor. In Anthropic’s case, the company said Claude can complete most of a given task 'end-to-end from a high-level prompt' while still being under human supervision. It is not fully autonomous yet, but the direction is clear.
Anthony Aguirre, president and CEO of the nonprofit Future of Life Institute and a physics professor at the University of California, Santa Cruz, said autonomous recursive self-improvement means AI that can improve itself by designing the next version of the system, 'then the next version, and so on.' He warned that as AI does more of the work, 'it gets faster, because AI operates just much, much more quickly than the humans do.'
Aguirre said the result of that progress is 'extremely scary' and called full autonomy 'the worst idea in the history of humanity' to do this. He said AI companies are getting closer to those bigger leaps in improvement, and that Anthropic’s public disclosure showed more research being done by AI over time.
John Thickstun, an assistant professor of computer science at Cornell University who studies methods that control the behavior of AI models, said the fear around recursive self-improvement is based largely on a runaway superintelligence emerging from the process. But he also said a more grounded view is that recursive self-improvement has been happening in AI development for years.
'We have already, for years, been using these models in supportive roles for creating the next version of these models,' Thickstun said. He added that people use past generations of models to write code for AI systems that then create the next generation.
OpenAI said in its announcement that while recursive self-improvement can help align models’ actions with human values and intentions, that does not mean 'rapid RSI is necessarily an outcome we should pursue.' The company said, 'Whether and how to proceed must depend on our ability to preserve human control and on informed democratic choices about the benefits and risks.'
Elon Musk, by contrast, appears more willing to push ahead. In March, he said that for xAI’s Grok models, 'humans are gradually getting less and less in the loop' on model improvement and that 'every successive model is built by the one before it.'
For capital markets, the lesson is simple: scale is attractive until it starts to outrun accountability. The same systems that promise gains in science and medicine also concentrate more power inside the machine itself, leaving less room for human oversight.
That is the real fault line. Free enterprise rewards innovation, but it also depends on clear rules and human responsibility. If AI firms are racing toward systems that improve themselves, the burden is on them to prove that control can keep pace with capability before the administrative state or a public panic tries to impose the answer for everyone.
