Actionable Attribution Maps for Scientific Machine Learning
It addresses the challenge for scientists in interpreting black-box models to gain insights that could lead to discoveries, but appears incremental as it builds on existing attribution and generative methods.
The paper tackles the problem of extracting actionable knowledge from opaque deep learning models in scientific domains by proposing techniques that inject domain-specific concepts as tunable knobs, enabling better understanding and actionable insights for scientists.
The scientific community has been increasingly interested in harnessing the power of deep learning to solve various domain challenges. However, despite the effectiveness in building predictive models, fundamental challenges exist in extracting actionable knowledge from the deep neural network due to their opaque nature. In this work, we propose techniques for exploring the behavior of deep learning models by injecting domain-specific actionable concepts as tunable ``knobs'' in the analysis pipeline. By incorporating the domain knowledge with generative modeling, we are not only able to better understand the behavior of these black-box models, but also provide scientists with actionable insights that can potentially lead to fundamental discoveries.