Anthropic Secretly Lobbying Government to Only Shut Down an AI Model If It's Poised to Kill More Than 50 People

The proposal from Anthropic suggests a specific threshold for intervention in AI model operations: a potential to cause over 50 deaths. This establishes a clear, albeit high, bar for regulatory action. The implication is that models not meeting this severe criterion would largely operate without external shutdown mandates, placing considerable responsibility and latitude with the AI developers themselves.

This approach highlights a tension in AI policy discussions between developer autonomy and public safety. By setting a very high bar for government-mandated shutdowns, Anthropic's proposal leans towards allowing AI labs significant control over their models' deployment and operation. It suggests a belief that industry self-regulation, or at least a high threshold for external intervention, is a viable path for managing AI risks.

The focus on a specific fatality count as the trigger for regulatory action offers a concrete, measurable metric, which could simplify enforcement compared to more subjective risk assessments. However, the challenge lies in accurately predicting or proving such a severe outcome before it occurs, especially given the complexities and emergent behaviors of advanced AI models. The scheme would require robust, verifiable methods for risk assessment that are not yet fully established or agreed upon.