Scale-Consistent State-Space Dynamics via Fractal of Stationary Transformations
This work addresses a structural limitation in deep learning for researchers and practitioners, offering a novel method to improve model interpretability and efficiency, though it appears incremental as it builds on existing state-space models.
The paper tackled the problem of deep learning models lacking structural guarantees for intermediate representations, which makes early stopping and adaptive computation ill-posed, by proposing Fractal of Stationary Transformations (FROST) to enforce scale-consistent latent dynamics, resulting in empirical verification on ImageNet-100 showing that adaptive efficiency emerges from the aligned latent geometry.
Recent deep learning models increasingly rely on depth without structural guarantees on the validity of intermediate representations, rendering early stopping and adaptive computation ill-posed. We address this limitation by formulating a structural requirement for state-space model's scale-consistent latent dynamics across iterative refinement, and derive Fractal of Stationary Transformations (FROST), which enforces a self-similar representation manifold through a fractal inductive bias. Under this geometry, intermediate states correspond to different resolutions of a shared representation, and we provide a geometric analysis establishing contraction and stable convergence across iterations. As a consequence of this scale-consistent structure, halting naturally admits a ranking-based formulation driven by intrinsic feature quality rather than extrinsic objectives. Controlled experiments on ImageNet-100 empirically verify the predicted scale-consistent behavior, showing that adaptive efficiency emerges from the aligned latent geometry.