Atomizer: Generalizing to new modalities by breaking satellite images down to a set of scalars
This addresses the challenge for remote sensing applications where existing models require retraining for new data configurations, offering a more adaptable solution, though it is incremental in improving generalization within a specific domain.
The paper tackles the problem of limited generalization across diverse remote sensing modalities by introducing Atomizer, a flexible architecture that represents images as sets of scalars with metadata, enabling a single encoder to handle arbitrary configurations without retraining, and it outperforms standard models in modality-disjoint evaluations with robust performance across resolutions and spatial sizes.
The growing number of Earth observation satellites has led to increasingly diverse remote sensing data, with varying spatial, spectral, and temporal configurations. Most existing models rely on fixed input formats and modality-specific encoders, which require retraining when new configurations are introduced, limiting their ability to generalize across modalities. We introduce Atomizer, a flexible architecture that represents remote sensing images as sets of scalars, each corresponding to a spectral band value of a pixel. Each scalar is enriched with contextual metadata (acquisition time, spatial resolution, wavelength, and bandwidth), producing an atomic representation that allows a single encoder to process arbitrary modalities without interpolation or resampling. Atomizer uses structured tokenization with Fourier features and non-uniform radial basis functions to encode content and context, and maps tokens into a latent space via cross-attention. Under modality-disjoint evaluations, Atomizer outperforms standard models and demonstrates robust performance across varying resolutions and spatial sizes.