Semi-Supervised Learning Enabled by Multiscale Deep Neural Network Inversion
This addresses the need for robust semi-supervised learning methods to reduce labeling costs in machine perception, though it appears incremental as it builds on existing semi-supervised techniques.
The paper tackled the problem of deep neural networks requiring large labeled datasets by developing a general semi-supervised learning loss function that works with any topology, achieving state-of-the-art performance with 9.82% test error on SVHN and 16.38% on CIFAR10.
Deep Neural Networks (DNNs) provide state-of-the-art solutions in several difficult machine perceptual tasks. However, their performance relies on the availability of a large set of labeled training data, which limits the breadth of their applicability. Hence, there is a need for new {\em semi-supervised learning} methods for DNNs that can leverage both (a small amount of) labeled and unlabeled training data. In this paper, we develop a general loss function enabling DNNs of any topology to be trained in a semi-supervised manner without extra hyper-parameters. As opposed to current semi-supervised techniques based on topology-specific or unstable approaches, ours is both robust and general. We demonstrate that our approach reaches state-of-the-art performance on the SVHN ($9.82\%$ test error, with $500$ labels and wide Resnet) and CIFAR10 (16.38% test error, with 8000 labels and sigmoid convolutional neural network) data sets.