The Propagation Field: A Geometric Substrate Theory of Deep Learning
For deep learning researchers, this work introduces a new geometric perspective on neural network internals that can improve model understanding and training, though it is currently demonstrated in controlled settings.
The paper proposes that neural networks should be understood through the geometry of their internal propagation (the propagation field) rather than just endpoint behavior. It shows that endpoint-equivalent models can have vastly different internal structures, and field-aware objectives improve generalization, robustness, and continual learning performance (e.g., on Split CIFAR-100, DER++ with field preservation improves average accuracy and backward transfer).
Modern deep learning treats neural networks primarily as endpoint functions from inputs to outputs. Inspired by the shift from force to geometry in physics, we ask whether a network should instead be understood through the geometry of its internal propagation. We define a neural propagation field as the collection of hidden-state trajectories and local Jacobian operators across depth. Endpoint losses constrain only the boundary behavior of this field, leaving its interior geometry underdetermined. We show that endpoint-equivalent models can differ by orders of magnitude in trajectory and Jacobian structure, and introduce observable field metrics such as path sensitivity, solver consistency, and trajectory/Jacobian retention. In controlled teacher-flow and PDE systems, endpoint fitting fails to recover the underlying propagation law. In real multi-path tasks, field-aware objectives improve unseen-path generalization, OOD robustness, and calibration when aligned with the observation structure, but can collapse when over-constrained. In continual learning, field-preservation regularization complements replay and distillation: on Split CIFAR-100, DER++ with field preservation improves average accuracy, backward transfer, and field-retention metrics. These results identify propagation-field quality as a measurable and trainable property of neural networks beyond endpoint performance.