Learning by Self-Explanation, with Application to Neural Architecture Search
This paper tackles the problem of improving machine learning model performance by introducing a self-explanation mechanism, which could be beneficial for researchers working on model interpretability and learning efficiency.
This paper proposes Learning by Self-Explanation (LeaSE), a machine learning method inspired by human learning, where an 'explainer' model improves its learning by explaining its predictions to an 'audience' model. The method is formulated as a four-level optimization problem and applied to neural architecture search, demonstrating effectiveness on CIFAR-100, CIFAR-10, and ImageNet.
Learning by self-explanation is an effective learning technique in human learning, where students explain a learned topic to themselves for deepening their understanding of this topic. It is interesting to investigate whether this explanation-driven learning methodology broadly used by humans is helpful for improving machine learning as well. Based on this inspiration, we propose a novel machine learning method called learning by self-explanation (LeaSE). In our approach, an explainer model improves its learning ability by trying to clearly explain to an audience model regarding how a prediction outcome is made. LeaSE is formulated as a four-level optimization problem involving a sequence of four learning stages which are conducted end-to-end in a unified framework: 1) explainer learns; 2) explainer explains; 3) audience learns; 4) explainer re-learns based on the performance of the audience. We develop an efficient algorithm to solve the LeaSE problem. We apply LeaSE for neural architecture search on CIFAR-100, CIFAR-10, and ImageNet. Experimental results strongly demonstrate the effectiveness of our method.