Reversible and irreversible bracket-based dynamics for deep graph neural networks
This work addresses the problem of understanding and improving deep graph neural network training for researchers in machine learning and graph analysis, though it appears incremental as it builds on existing physics-inspired approaches.
The authors tackled the unclear role of physics-inspired architectures in training deep graph neural networks without oversmoothing, by proposing novel GNN architectures based on bracket-based dynamical systems that provably conserve energy or generate positive dissipation, leading to inherently explainable constructions that clarify the roles of reversibility and irreversibility.
Recent works have shown that physics-inspired architectures allow the training of deep graph neural networks (GNNs) without oversmoothing. The role of these physics is unclear, however, with successful examples of both reversible (e.g., Hamiltonian) and irreversible (e.g., diffusion) phenomena producing comparable results despite diametrically opposed mechanisms, and further complications arising due to empirical departures from mathematical theory. This work presents a series of novel GNN architectures based upon structure-preserving bracket-based dynamical systems, which are provably guaranteed to either conserve energy or generate positive dissipation with increasing depth. It is shown that the theoretically principled framework employed here allows for inherently explainable constructions, which contextualize departures from theory in current architectures and better elucidate the roles of reversibility and irreversibility in network performance.