'Less Than One'-Shot Learning: Learning N Classes From M<N Samples
This addresses the challenge of reducing training data requirements for deep neural networks, offering a novel approach to extreme few-shot learning.
The paper tackles the problem of learning N classes from M<N samples, proposing a 'less than one'-shot learning task and showing it is achievable using soft labels, with analysis of decision landscapes and theoretical lower bounds.
Deep neural networks require large training sets but suffer from high computational cost and long training times. Training on much smaller training sets while maintaining nearly the same accuracy would be very beneficial. In the few-shot learning setting, a model must learn a new class given only a small number of samples from that class. One-shot learning is an extreme form of few-shot learning where the model must learn a new class from a single example. We propose the `less than one'-shot learning task where models must learn $N$ new classes given only $M<N$ examples and we show that this is achievable with the help of soft labels. We use a soft-label generalization of the k-Nearest Neighbors classifier to explore the intricate decision landscapes that can be created in the `less than one'-shot learning setting. We analyze these decision landscapes to derive theoretical lower bounds for separating $N$ classes using $M<N$ soft-label samples and investigate the robustness of the resulting systems.