Quantitative Performance Assessment of CNN Units via Topological Entropy Calculation
This work addresses the challenge of understanding CNN mechanisms for researchers, but it is incremental as it builds on existing topological methods for unit analysis.
The authors tackled the problem of quantitatively assessing the status of individual units in convolutional neural networks (CNNs) by proposing a method using algebraic topological tools to calculate a feature entropy, which measures the chaos in spatial patterns and shows trends with layer depth and training loss, enabling discrimination between networks with different generalization abilities.
Identifying the status of individual network units is critical for understanding the mechanism of convolutional neural networks (CNNs). However, it is still challenging to reliably give a general indication of unit status, especially for units in different network models. To this end, we propose a novel method for quantitatively clarifying the status of single unit in CNN using algebraic topological tools. Unit status is indicated via the calculation of a defined topological-based entropy, called feature entropy, which measures the degree of chaos of the global spatial pattern hidden in the unit for a category. In this way, feature entropy could provide an accurate indication of status for units in different networks with diverse situations like weight-rescaling operation. Further, we show that feature entropy decreases as the layer goes deeper and shares almost simultaneous trend with loss during training. We show that by investigating the feature entropy of units on only training data, it could give discrimination between networks with different generalization ability from the view of the effectiveness of feature representations.