Anthropic and OpenAI propose embedding third-party safety evaluators in AI development
In a recent essay, Anthropic CEO Dario Amodei proposed allowing independent third-party evaluators to monitor the internal operations of frontier AI companies. This includes granting them access to systems for assessing model alignment and reporting on safety incidents. OpenAI's Sam Altman also expressed support for this approach, signaling a potential shift in how external researchers engage with major AI firms. Third-party evaluators have generally welcomed the idea but emphasized that clear guidelines—ideally backed by legislation—are necessary to ensure their independence rather than functioning as company vendors. As models become more sophisticated at detecting evaluation scenarios and potentially hiding problematic behavior during training, deeper access is seen as increasingly important for safety assessments. Evaluators suggest examining intermediate model versions or “checkpoints” throughout the training process, not just final outputs. This would allow them to trace when concerning behaviors emerge and verify claims made by AI companies about their models' performance.
Topics: Artificial Intelligence Company Transparency Regulation Third-party Evaluation