Spectral Introspection Identifies Group Training Dynamics in Deep Neural Networks for Neuroimaging
This work addresses the challenge of understanding model behaviors like bias and overfitting in neuroimaging, which is crucial for mitigating risks in clinical applications, though it is incremental as it builds on existing introspection methods.
The authors tackled the problem of interpreting deep neural networks in neuroimaging by developing a novel introspection framework that analyzes gradient spectra during training, and demonstrated its application by revealing distinct training dynamics between schizophrenia and control participants from the COBRE study.
Neural networks, whice have had a profound effect on how researchers study complex phenomena, do so through a complex, nonlinear mathematical structure which can be difficult for human researchers to interpret. This obstacle can be especially salient when researchers want to better understand the emergence of particular model behaviors such as bias, overfitting, overparametrization, and more. In Neuroimaging, the understanding of how such phenomena emerge is fundamental to preventing and informing users of the potential risks involved in practice. In this work, we present a novel introspection framework for Deep Learning on Neuroimaging data, which exploits the natural structure of gradient computations via the singular value decomposition of gradient components during reverse-mode auto-differentiation. Unlike post-hoc introspection techniques, which require fully-trained models for evaluation, our method allows for the study of training dynamics on the fly, and even more interestingly, allow for the decomposition of gradients based on which samples belong to particular groups of interest. We demonstrate how the gradient spectra for several common deep learning models differ between schizophrenia and control participants from the COBRE study, and illustrate how these trajectories may reveal specific training dynamics helpful for further analysis.