Interpretable Few-Shot Learning via Linear Distillation
This work addresses the need for interpretable models in few-shot learning, though it appears incremental as it builds on existing linear network methods.
The paper tackled the problem of improving the interpretability and performance of linear neural networks in few-shot learning by proposing Linear Distillation Learning, which uses linear functions per class to simulate a teacher network, resulting in better performance on MNIST and Omniglot datasets compared to classical Logistic Regression.
It is important to develop mathematically tractable models than can interpret knowledge extracted from the data and provide reasonable predictions. In this paper, we present a Linear Distillation Learning, a simple remedy to improve the performance of linear neural networks. Our approach is based on using a linear function for each class in a dataset, which is trained to simulate the output of a teacher linear network for each class separately. We tested our model on MNIST and Omniglot datasets in the Few-Shot learning manner. It showed better results than other interpretable models such as classical Logistic Regression.