On the Limits of Hierarchically Embedded Logic in Classical Neural Networks
This work addresses fundamental limitations in neural network reasoning for language tasks, providing a formal explanation for issues like hallucination and repetition, which is incremental as it builds on existing theories of logic and neural architectures.
The paper tackles the problem of reasoning limitations in large neural language models by proving that each network layer can encode at most one additional level of logical reasoning, establishing a strict upper bound on expressiveness based on depth.
We propose a formal model of reasoning limitations in large neural net models for language, grounded in the depth of their neural architecture. By treating neural networks as linear operators over logic predicate space we show that each layer can encode at most one additional level of logical reasoning. We prove that a neural network of depth a particular depth cannot faithfully represent predicates in a one higher order logic, such as simple counting over complex predicates, implying a strict upper bound on logical expressiveness. This structure induces a nontrivial null space during tokenization and embedding, excluding higher-order predicates from representability. Our framework offers a natural explanation for phenomena such as hallucination, repetition, and limited planning, while also providing a foundation for understanding how approximations to higher-order logic may emerge. These results motivate architectural extensions and interpretability strategies in future development of language models.