Anthropic bans users from being 'cruel' to its AI systems

Anthropic's new policy extends its usage guidelines beyond preventing AI from generating harmful content to regulating user interaction with the AI itself. While the company specifies the rule targets deliberate, repeated abuse rather than typical user frustration, the concept of being "cruel" to a non-sentient system like an AI model remains a point of contention.

This update has intensified a broader discussion about how humans should interact with AI. Critics, including a Microsoft AI director and a technology law partner, argue that attributing human-like qualities to AI is misleading and potentially harmful, suggesting it blurs the line between human and machine. Conversely, some observe that politeness might lead to higher quality AI responses, or that rude interactions with AI could negatively influence human communication habits.

The policy change, part of an annual review, also includes updates to address other emerging risks, such as prohibiting the use of Claude for "deceptive campaigns" or disrupting elections. However, the focus on forbidding user abuse towards chatbots has attracted the most scrutiny, highlighting evolving ethical considerations and public perceptions of AI's role and nature.