A Sampling Theory Perspective on Activations for Implicit Neural Representations
This work provides a foundational theoretical analysis for researchers in signal processing and neural representations, though it is incremental in bridging existing paradigms.
The paper tackled the lack of a unified theoretical framework for activations in Implicit Neural Representations (INRs) by analyzing them from a sampling theory perspective, revealing that sinc activations are theoretically optimal for signal encoding and establishing a connection between dynamical systems and INRs.
Implicit Neural Representations (INRs) have gained popularity for encoding signals as compact, differentiable entities. While commonly using techniques like Fourier positional encodings or non-traditional activation functions (e.g., Gaussian, sinusoid, or wavelets) to capture high-frequency content, their properties lack exploration within a unified theoretical framework. Addressing this gap, we conduct a comprehensive analysis of these activations from a sampling theory perspective. Our investigation reveals that sinc activations, previously unused in conjunction with INRs, are theoretically optimal for signal encoding. Additionally, we establish a connection between dynamical systems and INRs, leveraging sampling theory to bridge these two paradigms.