Distilling a Deep Neural Network into a Takagi-Sugeno-Kang Fuzzy Inference System
This addresses the need for explainable AI in classification tasks, though it is incremental as it builds on existing knowledge distillation methods.
The paper tackles the problem of deep neural networks being black boxes by distilling their knowledge into a Takagi-Sugeno-Kang fuzzy inference system to improve interpretability, with experiments showing the distilled model generalizes better than one trained directly from data.
Deep neural networks (DNNs) demonstrate great success in classification tasks. However, they act as black boxes and we don't know how they make decisions in a particular classification task. To this end, we propose to distill the knowledge from a DNN into a fuzzy inference system (FIS), which is Takagi-Sugeno-Kang (TSK)-type in this paper. The model has the capability to express the knowledge acquired by a DNN based on fuzzy rules, thus explaining a particular decision much easier. Knowledge distillation (KD) is applied to create a TSK-type FIS that generalizes better than one directly from the training data, which is guaranteed through experiments in this paper. To further improve the performances, we modify the baseline method of KD and obtain good results.