Revisiting a kNN-based Image Classification System with High-capacity Storage
This addresses the need for more interpretable and updatable image classification systems, though it is incremental as it revisits kNN with modern storage.
The paper tackles the problem of updating and interpreting knowledge in image classification systems by storing knowledge externally in high-capacity storage instead of in model parameters, achieving 79.8% top-1 accuracy on ImageNet and 90.8% on Split CIFAR-100 in incremental learning.
In existing image classification systems that use deep neural networks, the knowledge needed for image classification is implicitly stored in model parameters. If users want to update this knowledge, then they need to fine-tune the model parameters. Moreover, users cannot verify the validity of inference results or evaluate the contribution of knowledge to the results. In this paper, we investigate a system that stores knowledge for image classification, such as image feature maps, labels, and original images, not in model parameters but in external high-capacity storage. Our system refers to the storage like a database when classifying input images. To increase knowledge, our system updates the database instead of fine-tuning model parameters, which avoids catastrophic forgetting in incremental learning scenarios. We revisit a kNN (k-Nearest Neighbor) classifier and employ it in our system. By analyzing the neighborhood samples referred by the kNN algorithm, we can interpret how knowledge learned in the past is used for inference results. Our system achieves 79.8% top-1 accuracy on the ImageNet dataset without fine-tuning model parameters after pretraining, and 90.8% accuracy on the Split CIFAR-100 dataset in the task incremental learning setting.