Unveiling the Potential of Superexpressive Networks in Implicit Neural Representations
This work addresses the challenge of improving signal representation and solving inverse problems and PDEs in computer vision and scientific machine learning, but it appears incremental as it builds on an existing network structure.
The study tackled the problem of learning neural functions for representing complex signals and performing downstream tasks by evaluating superexpressive networks, showing they can surpass recent implicit neural representations that use specialized activation functions.
In this study, we examine the potential of one of the ``superexpressive'' networks in the context of learning neural functions for representing complex signals and performing machine learning downstream tasks. Our focus is on evaluating their performance on computer vision and scientific machine learning tasks including signal representation/inverse problems and solutions of partial differential equations. Through an empirical investigation in various benchmark tasks, we demonstrate that superexpressive networks, as proposed by [Zhang et al. NeurIPS, 2022], which employ a specialized network structure characterized by having an additional dimension, namely width, depth, and ``height'', can surpass recent implicit neural representations that use highly-specialized nonlinear activation functions.