EchoChain: A Full-Duplex Benchmark for State-Update Reasoning Under Interruptions
For developers of real-time voice assistants, EchoChain provides a controlled benchmark to diagnose and improve state-update reasoning under interruptions, a previously unaddressed failure mode.
EchoChain introduces a benchmark for evaluating full-duplex state-update reasoning under mid-speech interruptions, finding that no real-time voice model exceeds a 50% pass rate and that interruptions cause a 40.2% increase in failures compared to half-duplex controls.
Real-time voice assistants must revise task state when users interrupt mid-response, but existing spoken-dialog benchmarks largely evaluate turn-based interaction and miss this failure mode. We introduce EchoChain, a controlled benchmark for evaluating full-duplex state-update reasoning under mid-speech interruptions. EchoChain identifies three recurring failure patterns in post-interruption continuations: contextual inertia, interruption amnesia, and objective displacement. The benchmark generates scenario-driven conversations and injects interruptions at a standardized point relative to assistant speech onset, enabling controlled cross-model comparison. In a paired half-duplex control, total failures drop by 40.2% relative to interrupted runs, indicating that many errors are driven by state-update reasoning under interruption rather than task difficulty alone. Across evaluated real-time voice models, no system exceeds a 50% pass rate, showing substantial room for improvement in mid-generation state revision. EchoChain provides a reproducible benchmark for diagnosing state-update reasoning failures in full-duplex voice interaction.