FiRe: Frequency Reparameterization as a Preconditioner for Periodic Implicit Neural Representations
For practitioners of implicit neural representations, FiRe offers a simple plug-in improvement for periodic activations that accelerates optimization, though gains are incremental and budget-dependent.
FiRe reparameterizes per-neuron frequencies in periodic implicit neural representations via a low-rank gating path, acting as an implicit preconditioner that improves optimization conditioning and convergence speed. On 2D image fitting, it achieves up to +1 dB PSNR gain over baselines at short training budgets, with gains diminishing at full convergence.
Periodic Implicit Neural Representations (INRs) such as SIREN and FINER assign every neuron, the same global frequency, spending the representational budget inefficiently when local signal content varies. We introduce FiRe (Frequency Reparameterization), that accelerates optimization by reparameterizing per-neuron frequency of periodic INRs without changing their underlying activation function. FiRe gives each neuron a bounded, input-dependent frequency via a separate low-rank gating path and is applicable to any periodic activation function. The gate acts as an implicit preconditioner that improves optimization conditioning at initialization via the Neural Tangent Kernel (NTK). This better-conditioned initialization makes optimization converge faster, and the high-frequency content of the reconstruction tracks the target more closely at a fixed computational budget. On 2D image fitting, FiRe increases PSNR over a parameter-matched baseline (up to +1 dB at short training budgets), with gains that vary with resolution and diminish at full convergence. We characterize how performance depends on resolution, rank, and training budget, and give an NTK account that predicts these trends.