Concept-Monitor: Understanding DNN training through individual neurons
It addresses the lack of transparency in DNN training for researchers and practitioners, offering tools to monitor and enhance training, though it appears incremental in building on existing interpretability methods.
The paper tackles the problem of understanding DNN training processes by proposing Concept-Monitor, a framework that uses a unified embedding space and concept diversity metric to provide interpretable visualizations and indicators, and it shows that a training regularizer based on this improves performance.
In this work, we propose a general framework called Concept-Monitor to help demystify the black-box DNN training processes automatically using a novel unified embedding space and concept diversity metric. Concept-Monitor enables human-interpretable visualization and indicators of the DNN training processes and facilitates transparency as well as deeper understanding on how DNNs develop along the during training. Inspired by these findings, we also propose a new training regularizer that incentivizes hidden neurons to learn diverse concepts, which we show to improve training performance. Finally, we apply Concept-Monitor to conduct several case studies on different training paradigms including adversarial training, fine-tuning and network pruning via the Lottery Ticket Hypothesis