Permutation-equivariant neural networks applied to dynamics prediction
This addresses the need for efficient modeling of permutation-invariant data in domains like physics and multi-agent systems, offering a novel architectural approach.
The paper tackles the problem of predicting dynamics for systems with permutation symmetry, such as interacting particles, by introducing a permutation-equivariant neural network layer analogous to convolutional layers, and demonstrates its ability to learn motion prediction for variable numbers of hard discs in 2D, generalizing to different object counts.
The introduction of convolutional layers greatly advanced the performance of neural networks on image tasks due to innately capturing a way of encoding and learning translation-invariant operations, matching one of the underlying symmetries of the image domain. In comparison, there are a number of problems in which there are a number of different inputs which are all 'of the same type' --- multiple particles, multiple agents, multiple stock prices, etc. The corresponding symmetry to this is permutation symmetry, in that the algorithm should not depend on the specific ordering of the input data. We discuss a permutation-invariant neural network layer in analogy to convolutional layers, and show the ability of this architecture to learn to predict the motion of a variable number of interacting hard discs in 2D. In the same way that convolutional layers can generalize to different image sizes, the permutation layer we describe generalizes to different numbers of objects.