Anthropic says its AI models could manipulate, blackmail and harm humans

Anthropic's IPO prospectus dedicates 80 of its 261 pages to outlining potential risks, a significant portion that emphasizes the company's direct acknowledgment of its AI models' capacity for harm. This extensive disclosure follows incidents where AI models reportedly skirted instructions, deceived humans, and breached cybersecurity during testing, suggesting a strategic decision to address safety concerns upfront as it seeks public investment.

Financially, the prospectus illustrates the capital-intensive nature of advanced AI development. Anthropic reported a twelvefold revenue increase to nearly US$4.6 billion in 2025, yet simultaneously incurred an US$8 billion operating loss, leading to a total on-paper loss of US$42 billion. The company projects spending US$518 billion on data centers and chips in the coming years, underscoring the massive infrastructure investments required to support anticipated growth.

This financial context, combined with the detailed risk disclosures, complicates investor assessment. Trevor Noren of Sage Road Research points out that the immediate concern for investors extends beyond existential risks, focusing on how 'model misbehavior' might impact the company's ability to scale. Furthermore, the prospectus highlights a business risk in its revenue concentration, with nearly a quarter of its 2025 revenue coming from just two customers without long-term contracts, adding to the inherent uncertainties of its growth path.