SinBasis Networks: Matrix-Equivalent Feature Extraction for Wave-Like Optical Spectrograms
This provides a physics-informed deep learning approach for wave-form imaging modalities like optical spectrograms and audio mel-spectrograms, though it appears incremental as it builds on existing architectures.
The paper tackled the problem of extracting harmonic structures from wave-like images, which conventional feature extractors struggle with, by proposing SinBasis Networks that apply sinusoidal mappings to weight matrices in CNN, ViT, and Capsule architectures, resulting in substantial gains in reconstruction accuracy, translational robustness, and zero-shot cross-domain transfer across diverse datasets.
Wave-like images-from attosecond streaking spectrograms to optical spectra, audio mel-spectrograms and periodic video frames-encode critical harmonic structures that elude conventional feature extractors. We propose a unified, matrix-equivalent framework that reinterprets convolution and attention as linear transforms on flattened inputs, revealing filter weights as basis vectors spanning latent feature subspaces. To infuse spectral priors we apply elementwise $\sin(\cdot)$ mappings to each weight matrix. Embedding these transforms into CNN, ViT and Capsule architectures yields Sin-Basis Networks with heightened sensitivity to periodic motifs and built-in invariance to spatial shifts. Experiments on a diverse collection of wave-like image datasets-including 80,000 synthetic attosecond streaking spectrograms, thousands of Raman, photoluminescence and FTIR spectra, mel-spectrograms from AudioSet and cycle-pattern frames from Kinetics-demonstrate substantial gains in reconstruction accuracy, translational robustness and zero-shot cross-domain transfer. Theoretical analysis via matrix isomorphism and Mercer-kernel truncation quantifies how sinusoidal reparametrization enriches expressivity while preserving stability in data-scarce regimes. Sin-Basis Networks thus offer a lightweight, physics-informed approach to deep learning across all wave-form imaging modalities.