Robust MAE-Driven NAS: From Mask Reconstruction to Architecture Innovation
This work solves the data labeling bottleneck for researchers and practitioners in NAS, offering a robust unsupervised alternative, though it is incremental as it builds on existing MAE and DARTS frameworks.
The paper tackles the problem of Neural Architecture Search (NAS) requiring labeled data by proposing an unsupervised method based on Masked Autoencoders (MAE), which uses image reconstruction to discover architectures without compromising performance and generalization, and addresses performance collapse in DARTS with a hierarchical decoder.
Neural Architecture Search (NAS) relies heavily on labeled data, which is labor-intensive and time-consuming to obtain. In this paper, we propose a novel NAS method based on an unsupervised paradigm, specifically Masked Autoencoders (MAE), thereby eliminating the need for labeled data. By replacing the supervised learning objective with an image reconstruction task, our approach enables the efficient discovery of network architectures without compromising performance and generalization ability. Additionally, we address the problem of performance collapse encountered in the widely-used Differentiable Architecture Search (DARTS) in the unsupervised setting by designing a hierarchical decoder. Extensive experiments across various datasets demonstrate the effectiveness and robustness of our method, offering empirical evidence of its superiority over the counterparts.