DenoMAE2.0: Improving Denoising Masked Autoencoders by Classifying Local Patches
This addresses challenges of high noise and data scarcity in wireless communication, representing an incremental enhancement to existing denoising masked autoencoder methods.
The paper tackles the problem of representation learning and robustness in semi-supervised learning for wireless communication by introducing DenoMAE2.0, which integrates local patch classification with reconstruction loss, achieving improvements of 1.1% over its predecessor on their dataset and 11.83-16.55% on the RadioML benchmark for modulation signal classification.
We introduce DenoMAE2.0, an enhanced denoising masked autoencoder that integrates a local patch classification objective alongside traditional reconstruction loss to improve representation learning and robustness. Unlike conventional Masked Autoencoders (MAE), which focus solely on reconstructing missing inputs, DenoMAE2.0 introduces position-aware classification of unmasked patches, enabling the model to capture fine-grained local features while maintaining global coherence. This dual-objective approach is particularly beneficial in semi-supervised learning for wireless communication, where high noise levels and data scarcity pose significant challenges. We conduct extensive experiments on modulation signal classification across a wide range of signal-to-noise ratios (SNRs), from extremely low to moderately high conditions and in a low data regime. Our results demonstrate that DenoMAE2.0 surpasses its predecessor, Deno-MAE, and other baselines in both denoising quality and downstream classification accuracy. DenoMAE2.0 achieves a 1.1% improvement over DenoMAE on our dataset and 11.83%, 16.55% significant improved accuracy gains on the RadioML benchmark, over DenoMAE, for constellation diagram classification of modulation signals.