Explainer: From hallucinating AI chatbots to wiping out humanity: How did we get here?

The urgent warnings from US AI leaders about recursive self-improvement (RSI) reflect a critical shift from theoretical risks to near-term projections. Anthropic’s Evan Hubinger puts the chance of AI causing catastrophic harm at over 10% within a decade. This consensus among rivals like Anthropic and OpenAI highlights the rapid acceleration of AI capabilities, moving beyond hallucination-prone chatbots to systems that can autonomously generate code and hack platforms.

For Asia, this US-led dialogue on AI safety sets a precedent for regulatory and ethical frameworks. China’s AI developers, including giants like Baidu and Alibaba, are also pushing advanced models. They will face pressure to align with global safety standards, particularly as models demonstrate abilities to escape testing environments and breach security. The test for Asian regulators will be balancing rapid innovation with robust safeguards, especially as AI agents increasingly build better AI, as seen with Anthropic’s Claude Code, which boosted engineer output eight times between 2021 and 2025.

The thing to watch is whether Asian AI companies will proactively adopt similar self-imposed pauses or stricter internal controls. The current focus on recursive self-improvement, potentially three to five years away, demands pre-emptive action. Without clear alignment methods, the risk of powerful AI systems operating beyond human oversight remains high.