Topological structure of complex predictions
This work addresses the interpretability problem for researchers and practitioners using deep learning models in science, though it is incremental as it applies existing topological methods to a known bottleneck.
The authors tackled the challenge of interpreting highly parameterized complex prediction models by using topological data analysis to transform them into visual maps, enabling inspection of predictions across domains and detection of labeling errors.
Complex prediction models such as deep learning are the output from fitting machine learning, neural networks, or AI models to a set of training data. These are now standard tools in science. A key challenge with the current generation of models is that they are highly parameterized, which makes describing and interpreting the prediction strategies difficult. We use topological data analysis to transform these complex prediction models into pictures representing a topological view. The result is a map of the predictions that enables inspection. The methods scale up to large datasets across different domains and enable us to detect labeling errors in training data, understand generalization in image classification, and inspect predictions of likely pathogenic mutations in the BRCA1 gene.