A Differential Manifold Perspective and Universality Analysis of Continuous Attractors in Artificial Neural Networks
This work addresses a foundational problem in neural network theory by providing a generalizable framework for understanding continuous attractors, which is incremental in building on existing mathematical connections.
The study tackled the lack of a unified framework for analyzing continuous attractors in artificial neural networks by establishing a novel differential manifold perspective, verifying compatibility with prior conclusions and demonstrating universality of singular value stratification in common models and datasets.
Continuous attractors are critical for information processing in both biological and artificial neural systems, with implications for spatial navigation, memory, and deep learning optimization. However, existing research lacks a unified framework to analyze their properties across diverse dynamical systems, limiting cross-architectural generalizability. This study establishes a novel framework from the perspective of differential manifolds to investigate continuous attractors in artificial neural networks. It verifies compatibility with prior conclusions, elucidates links between continuous attractor phenomena and eigenvalues of the local Jacobian matrix, and demonstrates the universality of singular value stratification in common classification models and datasets. These findings suggest continuous attractors may be ubiquitous in general neural networks, highlighting the need for a general theory, with the proposed framework offering a promising foundation given the close mathematical connection between eigenvalues and singular values.