Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors
This addresses a critical bottleneck in deep GNNs for graph learning tasks, offering a theoretical and practical solution to oversmoothing.
The paper tackles the problem of oversmoothing in deep Graph Neural Networks (GNNs) by reframing it from a bifurcation theory perspective, proving that replacing monotone activations like ReLU with a specific class of functions destabilizes the homogeneous state and creates stable, non-homogeneous patterns that resist oversmoothing, with experimental validation of a predicted scaling law.
Graph Neural Networks (GNNs) learn node representations through iterative network-based message-passing. While powerful, deep GNNs suffer from oversmoothing, where node features converge to a homogeneous, non-informative state. We re-frame this problem of representational collapse from a \emph{bifurcation theory} perspective, characterizing oversmoothing as convergence to a stable ``homogeneous fixed point.'' Our central contribution is the theoretical discovery that this undesired stability can be broken by replacing standard monotone activations (e.g., ReLU) with a class of functions. Using Lyapunov-Schmidt reduction, we analytically prove that this substitution induces a bifurcation that destabilizes the homogeneous state and creates a new pair of stable, non-homogeneous \emph{patterns} that provably resist oversmoothing. Our theory predicts a precise, nontrivial scaling law for the amplitude of these emergent patterns, which we quantitatively validate in experiments. Finally, we demonstrate the practical utility of our theory by deriving a closed-form, bifurcation-aware initialization and showing its utility in real benchmark experiments.