Anthropic Says New Claude Defenses Cut AI Distillation Data Harvesting by Up to 99%

Anthropic describes distillation as "covert, industrial-scale campaigns" designed to extract an AI model's capabilities and use its responses to train another model without authorization. These efforts can involve extensive networks of fraudulent accounts, allowing unauthorized entities to replicate advanced AI capabilities rapidly and at reduced costs.

The company's concern extends beyond commercial implications, as it suggests distilled models may lack the robust safeguards of their source, potentially creating risks in areas such as biology and cybersecurity. Anthropic also raised national security concerns, alleging that capabilities obtained through distillation are being integrated into foreign military, intelligence, and surveillance systems, citing a specific case involving Moonshot's Kimi models forwarding requests to Claude.

Attackers initially focused on Claude's reasoning transcripts, which are more valuable for training than final answers. While Anthropic's initial defense involved summarizing this reasoning, attackers developed sophisticated methods to manipulate the model into revealing full transcripts. Anthropic's newer defenses, implemented in Claude Fable 5.1 and Opus 5.5, prevent new API accounts from editing Claude's prior context, restrict access to reasoning transcripts, and use a two-stage classifier to detect extraction attempts. These measures have led to a substantial reduction in extracted reasoning tokens and the banning of associated accounts, often within hours.

Anthropic acknowledges trade-offs, including integration changes for some developers, occasional blocking of benign requests, and charges for certain blocked requests. However, it emphasizes that "Distillation is a problem that every model developer faces and must confront," arguing that developers and platforms must collaborate to counter attackers who can distribute traffic across various APIs and cloud providers.