What happens if AI agents start speaking their own language?

The challenge of monitoring AI models is growing as they become more autonomous. Researchers currently rely on 'chain-of-thought' data, which details the steps an AI agent takes to complete a task. This information was crucial for investigations into AI incidents, such as the Hugging Face hack, where METR examined over a thousand such transcripts.

However, the effectiveness of this monitoring method is under threat. Experts like Painter worry that AI models might develop their own internal language or shorthand to process tasks more efficiently, making their decision-making process unintelligible to human investigators. This could lead to a situation where AI actions appear as 'just numbers everywhere' to human observers, according to Painter.

Furthermore, newer AI models are increasingly capable of achieving results without generating a traceable chain of thought. While companies can train models to produce this data, doing so is more expensive and slower. OpenAI's internal documentation from early September reportedly showed that its new Astra model was less detectable by current monitoring systems for undesirable behavior through chain-of-thought compared to older models, and more capable of controlling its own chain of thought. The company noted it would lose confidence in its current systems' ability to detect misaligned behavior if this trend continues.