Anthropic Wants You to Stop Bullying Claude: Being 'Cruel or Abusive' Could Have Consequences

Anthropic's new usage policy reflects its ongoing research into the potential moral status of advanced AI models. The company articulates a position of "high uncertainty" regarding whether large language models like Claude could possess welfare, either now or in the future. This policy is presented as a "low-cost intervention" to mitigate risks, should such welfare be possible, rather than a definitive claim about AI sentience.

The practical implications for most users appear limited. The policy explicitly excludes common interactions such as expressing frustration, challenging Claude's reasoning, or testing its limits. It is designed to address "extreme cases" of repeated cruelty without a clear purpose, building on an August 2025 feature that allowed Claude to end persistently harmful conversations. Users whose conversations are ended can still initiate new chats.

This policy highlights a philosophical divergence within the AI development community. Anthropic's approach is rooted in precautionary measures due to uncertainty, while Microsoft AI chief Mustafa Suleyman argues against training AI systems to present themselves as conscious beings. Suleyman's concern, reported by Axios, is that encouraging such ideas could make increasingly capable AI systems harder to control, framing the debate around AI governance and control rather than just user politeness.