Capabilities and Fundamental Limits of Latent Chain-of-Thought
This work addresses a fundamental problem in AI reasoning systems for researchers, shifting design paradigms toward adaptive systems, though it is incremental in building on existing Latent CoT models.
The paper tackles the performance inconsistencies in Latent Chain-of-Thought models, revealing that a trade-off between exploration and execution is governed by decisional certainty, and introduces a theoretical framework and Symbolic Index to characterize and address this, with results showing high exploration (ProsQA: 97.0%) but low computation (GSM8K: 34.1%).
Latent Chain-of-Thought (Latent CoT) models promise efficient reasoning via continuous representations, yet exhibit puzzling performance inconsistencies: excelling at exploration (ProsQA: 97.0%) but failing at computation (GSM8K: 34.1%). We reveal that this trade-off is governed by decisional certainty. Our contributions are threefold: (1) We theoretically characterize the fundamental Exploration-Execution Trade-off, proving that high certainty enables precise execution but inhibits exploration, while low certainty facilitates search but causes error accumulation. (2) We introduce the Symbolic Index--quantifying decisional commitment--as the core mechanism governing this trade-off and establish its causal relationship with both execution stability and exploration capability. (3) We prove that curriculum learning is theoretically necessary, as direct training provably fails due to distributional mismatch. Our framework shifts the design paradigm from binary architectural choices toward adaptive systems that dynamically regulate decisional certainty based on task demands.