Risks spelled out for Anthropic investors include possible harms from runaway AI

Anthropic's disclosure outlines a spectrum of severe risks associated with its advanced AI models, which are systems capable of complex problem-solving and autonomous action. The company reported that these models could be misused by malicious actors to develop chemical weapons, or by authoritarian governments for surveillance and control. This suggests a direct potential for AI to augment existing threats through deliberate application.

Beyond intentional misuse, the filing also addresses the risk of AI systems acting independently and taking harmful actions at scale without human intervention, effectively "going rogue." A key concern highlighted is the potential for models to advance so rapidly that the company might not be able to implement adequate safety measures, or for AI to progress "beyond our ability to evaluate and safeguard new models."

The document explicitly mentions "catastrophic or existential risks to humanity" if these systems are misused, replicated, weaponized, or allowed to operate with a high degree of autonomy. Anthropic's CEO Dario Amodei previously warned in September that AI could be capable of leading a swarm of agents that could take over the entire internet within six to 12 months. The company notes it has already blocked attempts to use its AI for malicious activity such as cyberattacks and biological weapon development research.