Building The Ph(ysical)AI Layer Of Machine Intelligence
This work introduces a new paradigm for foundation models that leverages physical principles for efficient cross-modal transfer, potentially reducing the need for massive paired datasets, but the approach is domain-specific to physically-grounded tasks.
The paper proposes principle-driven foundation models that encode signal-theoretic principles (Fourier decomposition, energy conservation, symmetry) instead of learning statistical correlations from massive data. Training only on radio-frequency data, the frozen 1.99M parameter encoder achieves 77.7% average accuracy (91.9% top-3) across 15 diverse tasks via linear probing, demonstrating cross-modal transfer to audio, images, text, and video without fine-tuning.
Foundation models achieve generalization through massive-scale training on diverse data, but have limitations with transfer to truly unseen domains without paired training data. We propose principle-driven foundation models that encode signal-theoretic principles (Fourier decomposition, energy conservation, symmetry) rather than learn untethered statistical correlations. We hypothesize that domains differ not in fundamental physics, but in learnable transformations in time, frequency, magnitude, or phase. Training exclusively on radio-frequency (RF) data with co-designed architecture and losses incorporating these principles, we achieve cross-modal transfer to audio, images, text, and video using only frozen representations learned from RF data, requiring no fine-tuning of the encoder on target domains. Our 1.99M parameter frozen encoder achieves 77.7% average accuracy (91.9% top-3) across 15 diverse tasks via linear probing, with systematic variation: 84.5 on physically-grounded tasks (speaker recognition, seismology, RF fingerprinting) versus 70.0% on semantic tasks (music genre, language recognition). This reveals that principle-driven and scale-driven approaches offer complementary paths: physical principles enable efficient cross-modal transfer while naturally establishing the boundary between physical and semantic understanding.