Race for AI Self-Improvement Heats Up Amid Calls for Global Pause

British mathematician Jack Good coined the term “intelligence explosion” 61 years ago to describe what would happen when an intelligent machine entered a runaway cycle of fully automated self-improvement, quickly leaving human intelligence far behind. For decades, that hypothetical capability – often described as “recursive self-improvement” (RSI) – has been seen as AI’s holy grail. The logic goes that the first country or company to achieve RSI would leave its competitors in the dust, cementing an unassailable lead. The company has called for “the option” to temporarily pause AI development globally, citing the heightened risks of losing control of AI systems once RSI is achieved – though such warnings have also been criticised as marketing hype. Anthropic is not the only one trying to automate AI research and development (R&D). In China, leading AI companies and researchers are also beginning to broach the subject of self-improving AI systems. “We’ve already begun to see an implementable and actionable path to achieve AI model self-evolution,” she told a room packed with major policymakers and investors.