Improved Knowledge Distillation for Land-Use Image Classification
For practitioners needing efficient land-use classification models, this work offers a modest improvement over existing distillation methods, though it is incremental in nature.
An improved Knowledge Distillation framework combining KL divergence and Cosine Similarity losses achieves 99.04% accuracy on land-use classification, outperforming baseline student training and single-loss distillation while compressing VGG16 into MobileNetV2.
In the present article, an improved Knowledge Distillation (KD) framework has been proposed for efficient compression of deep convolutional neural networks for land-use image classification task. Motivated by the need to achieve competitive classification accuracy while reducing computational complexity, a teacher-student learning paradigm is adopted in which a VGG16 network transfers knowledge to a lightweight MobileNetV2 model. The proposed framework integrates hard supervision from ground truth labels with a soft supervision strategy that combines Kullback-Leibler divergence and Cosine Similarity losses. Experiments conducted on three land-use datasets show that the proposed KD-based method yields improved performance, and achieves an accuracy of 99.04%, outperforming both baseline student training and single-loss distillation approaches, while retaining substantial model compression.