OpenAI details new text watermarking system for ChatGPT, Codex, and the API

OpenAI's new text watermarking system, textGrain, is designed to identify AI-generated content by subtly influencing word choices in a way that is statistically detectable. This approach mirrors a method previously adopted by Anthropic for its Claude model, indicating a growing trend among major AI providers to address the identifiability of AI output, particularly in response to regulatory demands like the EU AI Act.

The rollout of textGrain will be phased, initially allowing API customers globally to opt-in for select models. For ChatGPT and Codex, the watermark will first apply to eligible users within the European Union. OpenAI also opened applications for access to its text watermark detector, which will initially be limited to approved researchers and expert organizations as the company works to evaluate and improve the technology.

However, OpenAI is transparent about the current limitations of its detection technology. It acknowledges the possibility of both false negatives and false positives, especially with shorter texts or content where word choice is constrained, such as in mathematics. Furthermore, the company notes that editing can significantly weaken the watermark; an evaluation of 400-token passages found that replacing 10% of words reduced detection from about 92% to 66%, and replacing 25% reduced it to 17%. This suggests that while watermarking offers a step towards transparency, it is not a definitive solution for identifying AI authorship or preventing misattribution.