Semi-MAE: Masked Autoencoders for Semi-supervised Vision Transformers
This addresses the challenge of limited labeled data for semi-supervised image classification, offering an incremental improvement over prior methods.
The paper tackles the problem of data scarcity in semi-supervised learning for Vision Transformers by proposing Semi-MAE, a framework that integrates a masked autoencoder branch to improve representation learning and pseudo-label accuracy, achieving 75.9% top-1 accuracy on ImageNet with 10% labels.
Vision Transformer (ViT) suffers from data scarcity in semi-supervised learning (SSL). To alleviate this issue, inspired by masked autoencoder (MAE), which is a data-efficient self-supervised learner, we propose Semi-MAE, a pure ViT-based SSL framework consisting of a parallel MAE branch to assist the visual representation learning and make the pseudo labels more accurate. The MAE branch is designed as an asymmetric architecture consisting of a lightweight decoder and a shared-weights encoder. We feed the weakly-augmented unlabeled data with a high masking ratio to the MAE branch and reconstruct the missing pixels. Semi-MAE achieves 75.9% top-1 accuracy on ImageNet with 10% labels, surpassing prior state-of-the-art in semi-supervised image classification. In addition, extensive experiments demonstrate that Semi-MAE can be readily used for other ViT models and masked image modeling methods.