SL$^{2}$A-INR: Single-Layer Learnable Activation for Implicit Neural Representation
This work addresses performance limitations in INR for vision-related domains, offering a novel architectural improvement that is incremental but impactful for specific applications.
The paper tackles the challenge of Implicit Neural Representation (INR) in capturing high-frequency components and diverse signal types by proposing SL$^{2}$A-INR, a hybrid network with a single-layer learnable activation function combined with ReLU MLP, which sets new benchmarks in accuracy, quality, and robustness across tasks like image representation, 3D shape reconstruction, and novel view synthesis.
Implicit Neural Representation (INR), leveraging a neural network to transform coordinate input into corresponding attributes, has recently driven significant advances in several vision-related domains. However, the performance of INR is heavily influenced by the choice of the nonlinear activation function used in its multilayer perceptron (MLP) architecture. To date, multiple nonlinearities have been investigated, but current INRs still face limitations in capturing high-frequency components and diverse signal types. We show that these challenges can be alleviated by introducing a novel approach in INR architecture. Specifically, we propose SL$^{2}$A-INR, a hybrid network that combines a single-layer learnable activation function with an MLP that uses traditional ReLU activations. Our method performs superior across diverse tasks, including image representation, 3D shape reconstruction, and novel view synthesis. Through comprehensive experiments, SL$^{2}$A-INR sets new benchmarks in accuracy, quality, and robustness for INR. Our Code is publicly available on~\href{https://github.com/Iceage7/SL2A-INR}{\textcolor{magenta}{GitHub}}.