Interpretable Geometric Deep Learning via Learnable Randomness Injection
This addresses the need for interpretable models in scientific applications like high-energy physics and biochemistry, where deploying complex GDL models is challenging due to lack of transparency, though it is incremental as it builds on existing GDL backbones.
The paper tackles the problem of interpretability in geometric deep learning (GDL) models for point cloud data by proposing learnable randomness injection (LRI), a mechanism that builds inherently interpretable models to detect informative points, resulting in better alignment with ground-truth patterns and improved robustness to distribution shifts.
Point cloud data is ubiquitous in scientific fields. Recently, geometric deep learning (GDL) has been widely applied to solve prediction tasks with such data. However, GDL models are often complicated and hardly interpretable, which poses concerns to scientists who are to deploy these models in scientific analysis and experiments. This work proposes a general mechanism, learnable randomness injection (LRI), which allows building inherently interpretable models based on general GDL backbones. LRI-induced models, once trained, can detect the points in the point cloud data that carry information indicative of the prediction label. We also propose four datasets from real scientific applications that cover the domains of high-energy physics and biochemistry to evaluate the LRI mechanism. Compared with previous post-hoc interpretation methods, the points detected by LRI align much better and stabler with the ground-truth patterns that have actual scientific meanings. LRI is grounded by the information bottleneck principle, and thus LRI-induced models are also more robust to distribution shifts between training and test scenarios. Our code and datasets are available at \url{https://github.com/Graph-COM/LRI}.