OmniField: Conditioned Neural Fields for Robust Multimodal Spatiotemporal Learning
This work addresses robust multimodal learning for real-world experimental data, offering an incremental improvement in handling noisy and incomplete modalities.
The paper tackled the problem of multimodal spatiotemporal learning with sparse, noisy, and varying modality data by proposing OmniField, a neural field framework that achieved consistent outperformance over eight baselines and maintained robust performance under heavy simulated sensor noise.
Multimodal spatiotemporal learning on real-world experimental data is constrained by two challenges: within-modality measurements are sparse, irregular, and noisy (QA/QC artifacts) but cross-modally correlated; the set of available modalities varies across space and time, shrinking the usable record unless models can adapt to arbitrary subsets at train and test time. We propose OmniField, a continuity-aware framework that learns a continuous neural field conditioned on available modalities and iteratively fuses cross-modal context. A multimodal crosstalk block architecture paired with iterative cross-modal refinement aligns signals prior to the decoder, enabling unified reconstruction, interpolation, forecasting, and cross-modal prediction without gridding or surrogate preprocessing. Extensive evaluations show that OmniField consistently outperforms eight strong multimodal spatiotemporal baselines. Under heavy simulated sensor noise, performance remains close to clean-input levels, highlighting robustness to corrupted measurements.