Diffusion Operator Geometry of Feedforward Representations
For researchers studying neural network representation geometry, this provides a more stable alternative to discrete curvature methods, though the contribution is incremental as it extends existing operator-based ideas to feedforward networks.
The paper develops a smooth operator-theoretic framework using Gaussian-kernel diffusion Markov operators to analyze feedforward representation geometry, deriving closed-form class affinities and spectra for Gaussian models. It shows that operator observables vary smoothly under perturbations, unlike discrete graph-based diagnostics, and validates the approach on synthetic and MNIST data.
Neural networks transform data through learned representations whose geometry affects separation, contraction, and generalization. Recent work studies this geometry using discrete curvature on neighborhood graphs, suggesting Ricci-flow-like behavior across layers. We develop a smooth operator-theoretic alternative for feedforward representation snapshots. Each feature cloud induces a Gaussian-kernel diffusion Markov operator, and transport, spectral, label-boundary, and local-scale observables are derived from this single object via Bakry-Emery $Γ$-calculus. In a balanced Gaussian class-conditional snapshot model with shared covariance, the population operator has closed-form class affinities, leakage, and coarse spectra, all controlled by pairwise regularized Mahalanobis separations $c_\varepsilon^{(a,b)}$. We also prove that the resulting operator observables vary smoothly under feature perturbations, while hard neighborhood-graph diagnostics can change discontinuously. Synthetic experiments validate the closed-form Gaussian bridge, while learned MNIST experiments show that the same operator observables track training, width, and perturbation stability. Together, these results give a stable operator-geometric framework for analyzing feedforward representation geometry.