On the Limits of Self-Improving in Large Language Models: The Singularity Is Not Near Without Symbolic Model Synthesis
This addresses the problem of autonomous self-improvement in AI for researchers and developers, showing that current methods are incremental and limited without external grounding.
The paper formalizes recursive self-training in LLMs as a dynamical system and proves that without persistent external grounding, it leads to degenerative dynamics like entropy decay and variance amplification, causing collapse. It proposes neurosymbolic integration using algorithmic probability to overcome these limits by identifying generative mechanisms instead of correlations.
We formalise recursive self-training in Large Language Models (LLMs) and Generative AI as a discrete-time dynamical system. We prove that if the proportion of exogenous, externally grounded signal $α_t$ vanishes asymptotically ($α_t \to 0$), the system undergoes degenerative dynamics. We derive two fundamental failure modes: (1) \textit{Entropy Decay}, where finite sampling effects induce monotonic loss of distributional diversity, and (2) \textit{Variance Amplification}, where the absence of persistent grounding causes distributional drift via a random-walk mechanism. These behaviours are architectural invariants of distributional learning on finite samples. We show that the collapse results apply specifically to closed-loop density matching without persistent external signal. Systems with non-vanishing exogenous grounding fall outside this regime. However, mainstream Singularity, AGI, and ASI narratives typically posit systems that become increasingly autonomous and require little to no human or external intervention for self-improvement. In that autonomy regime, the vanishing-signal condition is satisfied, and collapse follows under KL-based objectives. To overcome these limits, we propose neurosymbolic integration based on algorithmic probability and program synthesis. The Coding Theorem Method (CTM) enables identification of generative mechanisms rather than mere correlations, escaping the distribution-only constraints that bind standard statistical learning. We conclude that fully autonomous recursive density matching leads to degenerative fixed points, whereas externally anchored or mechanism-based approaches operate under fundamentally different asymptotic dynamics.