A Concrete Roadmap towards Safety Cases based on Chain-of-Thought Monitoring
This addresses safety concerns for advanced AI systems, but it is incremental as it outlines a research agenda rather than presenting new empirical results.
The paper tackles the problem of ensuring AI safety as systems approach dangerous capabilities by proposing a roadmap for constructing safety cases based on chain-of-thought monitoring, arguing it might support control and trustworthiness while analyzing threats like neuralese and encoded reasoning.
As AI systems approach dangerous capability levels where inability safety cases become insufficient, we need alternative approaches to ensure safety. This paper presents a roadmap for constructing safety cases based on chain-of-thought (CoT) monitoring in reasoning models and outlines our research agenda. We argue that CoT monitoring might support both control and trustworthiness safety cases. We propose a two-part safety case: (1) establishing that models lack dangerous capabilities when operating without their CoT, and (2) ensuring that any dangerous capabilities enabled by a CoT are detectable by CoT monitoring. We systematically examine two threats to monitorability: neuralese and encoded reasoning, which we categorize into three forms (linguistic drift, steganography, and alien reasoning) and analyze their potential drivers. We evaluate existing and novel techniques for maintaining CoT faithfulness. For cases where models produce non-monitorable reasoning, we explore the possibility of extracting a monitorable CoT from a non-monitorable CoT. To assess the viability of CoT monitoring safety cases, we establish prediction markets to aggregate forecasts on key technical milestones influencing their feasibility.