$μ$DARTS: Model Uncertainty-Aware Differentiable Architecture Search
This work addresses the need for more robust and reliable neural architectures in machine learning, particularly for applications sensitive to uncertainty, but it is incremental as it builds upon existing DARTS methods.
The paper tackles the problem of optimizing neural networks for high accuracy and low uncertainty by introducing a model uncertainty-aware differentiable architecture search method, resulting in improved accuracy and reduced uncertainty on datasets like CIFAR10 and ImageNet compared to existing DARTS methods, with higher robustness to noise.
We present a Model Uncertainty-aware Differentiable ARchiTecture Search ($μ$DARTS) that optimizes neural networks to simultaneously achieve high accuracy and low uncertainty. We introduce concrete dropout within DARTS cells and include a Monte-Carlo regularizer within the training loss to optimize the concrete dropout probabilities. A predictive variance term is introduced in the validation loss to enable searching for architecture with minimal model uncertainty. The experiments on CIFAR10, CIFAR100, SVHN, and ImageNet verify the effectiveness of $μ$DARTS in improving accuracy and reducing uncertainty compared to existing DARTS methods. Moreover, the final architecture obtained from $μ$DARTS shows higher robustness to noise at the input image and model parameters compared to the architecture obtained from existing DARTS methods.