FICNN: A Framework for the Interpretation of Deep Convolutional Neural Networks
This work addresses the need for better understanding of CNN models in machine learning, but it is incremental as it organizes existing knowledge rather than introducing new methods.
The paper tackles the problem of interpreting internal representations in deep convolutional neural networks (CNNs) by proposing a framework that distinguishes interpretation from explanation, defines six factors to characterize methods, and identifies unexplored areas in current research.
With the continue development of Convolutional Neural Networks (CNNs), there is a growing concern regarding representations that they encode internally. Analyzing these internal representations is referred to as model interpretation. While the task of model explanation, justifying the predictions of such models, has been studied extensively; the task of model interpretation has received less attention. The aim of this paper is to propose a framework for the study of interpretation methods designed for CNN models trained from visual data. More specifically, we first specify the difference between the interpretation and explanation tasks which are often considered the same in the literature. Then, we define a set of six specific factors that can be used to characterize interpretation methods. Third, based on the previous factors, we propose a framework for the positioning of interpretation methods. Our framework highlights that just a very small amount of the suggested factors, and combinations thereof, have been actually studied. Consequently, leaving significant areas unexplored. Following the proposed framework, we discuss existing interpretation methods and give some attention to the evaluation protocols followed to validate them. Finally, the paper highlights capabilities of the methods in producing feedback for enabling interpretation and proposes possible research problems arising from the framework.