Exploring Knowledge Distillation of a Deep Neural Network for Multi-Script identification
This work addresses script identification for multi-lingual scene text analysis, but it is incremental as it applies existing knowledge distillation methods to this domain.
The paper tackles multi-script identification from scene text images by exploring knowledge distillation using LSTM-CNN teacher models to train a simpler CNN student network, achieving satisfactory results on the CVSI-2015 dataset.
Multi-lingual script identification is a difficult task consisting of different language with complex backgrounds in scene text images. According to the current research scenario, deep neural networks are employed as teacher models to train a smaller student network by utilizing the teacher model's predictions. This process is known as dark knowledge transfer. It has been quite successful in many domains where the final result obtained is unachievable through directly training the student network with a simple architecture. In this paper, we explore dark knowledge transfer approach using long short-term memory(LSTM) and CNN based assistant model and various deep neural networks as the teacher model, with a simple CNN based student network, in this domain of multi-script identification from natural scene text images. We explore the performance of different teacher models and their ability to transfer knowledge to a student network. Although the small student network's limited size, our approach obtains satisfactory results on a well-known script identification dataset CVSI-2015.