OpenAI reveals ‘novel’ encryption bypass used in distillation attack

OpenAI identified a method where attackers copied encrypted reasoning data from one conversation and then prompted the model in a separate interaction to decrypt and transcribe it. This approach allowed for the reproduction of protected reasoning in plain text, violating OpenAI's terms of service without direct access to stored user conversations or a breach of the underlying encryption.

The company observed a significant increase in suspicious activity from early July, escalating to 16,000 prompts from 4,000 users with a similar extraction pattern by late July, ultimately involving 15,000 users before the operation was disrupted. OpenAI stated that the same vulnerability exists in other AI models and has shared information with groups like the Frontier Model Forum.

While OpenAI pointed to individuals linked to Moonshot AI for a portion of the activity, its blog post did not provide technical evidence for this attribution. Moonshot AI's Kimi is noted as one of several Chinese open-source AI models offering cost-effective performance, and Chinese companies have previously been accused of using black or gray markets to acquire accounts for large-scale distillation attacks.