Few-Shot Non-Parametric Learning with Deep Latent Variable Model
This addresses few-shot learning challenges for domains with abundant unlabeled data but scarce labels, offering a novel approach that is not incremental.
The authors tackled the problem of few-shot learning with unknown data distributions and limited labeled data by proposing NPC-LV, a framework that uses unsupervised generative models to build compressors for classification without further training. It outperformed supervised methods on image classification datasets in low-data regimes and even beat semi-supervised methods on CIFAR-10.
Most real-world problems that machine learning algorithms are expected to solve face the situation with 1) unknown data distribution; 2) little domain-specific knowledge; and 3) datasets with limited annotation. We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV), a learning framework for any dataset with abundant unlabeled data but very few labeled ones. By only training a generative model in an unsupervised way, the framework utilizes the data distribution to build a compressor. Using a compressor-based distance metric derived from Kolmogorov complexity, together with few labeled data, NPC-LV classifies without further training. We show that NPC-LV outperforms supervised methods on all three datasets on image classification in low data regime and even outperform semi-supervised learning methods on CIFAR-10. We demonstrate how and when negative evidence lowerbound (nELBO) can be used as an approximate compressed length for classification. By revealing the correlation between compression rate and classification accuracy, we illustrate that under NPC-LV, the improvement of generative models can enhance downstream classification accuracy.