PDE-CNNs: Axiomatic Derivations and Applications
This work addresses the need for more parameter-efficient and data-efficient neural networks in machine learning, though it appears incremental as it builds on existing PDE-G-CNN frameworks.
The authors tackled the problem of designing more efficient and accurate convolutional neural networks by deriving PDE-CNNs from desirable axioms, resulting in fewer parameters, increased accuracy, and better data efficiency compared to CNNs, as confirmed experimentally for small networks.
PDE-based Group Convolutional Neural Networks (PDE-G-CNNs) use solvers of evolution PDEs as substitutes for the conventional components in G-CNNs. PDE-G-CNNs can offer several benefits simultaneously: fewer parameters, inherent equivariance, better accuracy, and data efficiency. In this article we focus on Euclidean equivariant PDE-G-CNNs where the feature maps are two-dimensional throughout. We call this variant of the framework a PDE-CNN. From a machine learning perspective, we list several practically desirable axioms and derive from these which PDEs should be used in a PDE-CNN, this being our main contribution. Our approach to geometric learning via PDEs is inspired by the axioms of scale-space theory, which we generalize by introducing semifield-valued signals. Our theory reveals new PDEs that can be used in PDE-CNNs and we experimentally examine what impact these have on the accuracy of PDE-CNNs. We also confirm for small networks that PDE-CNNs offer fewer parameters, increased accuracy, and better data efficiency when compared to CNNs.