Transferability of Winning Lottery Tickets in Neural Network Differential Equation Solvers
This work incrementally extends lottery ticket research to neural differential equation solvers, potentially improving efficiency for scientific computing applications.
The authors investigated whether pruned neural network subnetworks (lottery tickets) found for Hamiltonian Neural Networks solving differential equations could transfer between different systems, finding that transferability depends on integration times and analyzing universality using renormalization group theory.
Recent work has shown that renormalisation group theory is a useful framework with which to describe the process of pruning neural networks via iterative magnitude pruning. This report formally describes the link between RG theory and IMP and extends previous results around the Lottery Ticket Hypothesis and Elastic Lottery Hypothesis to Hamiltonian Neural Networks for solving differential equations. We find lottery tickets for two Hamiltonian Neural Networks and demonstrate transferability between the two systems, with accuracy being dependent on integration times. The universality of the two systems is then analysed using tools from an RG perspective.