AI Researchers Warn Of Catastrophic Risks During New York City Council Hearing

The New York City Council hearing brought together prominent AI researchers and company executives to discuss potential catastrophic risks from advanced AI systems. Researchers like Alex Turner, Jacob Coxon, and Daniel Kokotajlo emphasized concerns about humanity losing control of AI and the rapid development pace without adequate safeguards. This suggests a shared apprehension among some experts regarding the urgency of AI safety, prompting legislative scrutiny.

The testimony highlighted a tension between the industry's development practices and calls for more robust oversight. While Google's Alice Friend backed a comprehensive federal framework, council officials pressed companies over their ability to quantify the risks of catastrophic AI incidents. Speaker Julie Menin criticized the industry's reliance on voluntary safeguards, indicating a gap between industry assurances and regulatory expectations concerning measurable AI safety.

The hearing's focus on "misaligned artificial intelligence" and the potential for AI to become an "adversary" reflects a shift in some expert discourse. Turner's comparison of AI risks to geopolitical concerns like China's AI ambitions underscores the perceived scale of the threat. This framing suggests the debate around AI safety is expanding beyond technical considerations to encompass broader societal and strategic implications, drawing attention from legislative bodies.