Diverse Explanations From Data-Driven and Domain-Driven Perspectives in the Physical Sciences
It addresses the challenge of interpretability for scientists using ML in fields like materials science, but it is incremental as it synthesizes existing debates and case studies without introducing new methods.
This paper tackles the problem of diverse and potentially inconsistent explanations from machine learning models in physical sciences, analyzing how different factors lead to varying interpretations and emphasizing the need for multiple perspectives and domain expertise to ensure trustworthy scientific outcomes.
Machine learning methods have been remarkably successful in material science, providing novel scientific insights, guiding future laboratory experiments, and accelerating materials discovery. Despite the promising performance of these models, understanding the decisions they make is also essential to ensure the scientific value of their outcomes. However, there is a recent and ongoing debate about the diversity of explanations, which potentially leads to scientific inconsistency. This Perspective explores the sources and implications of these diverse explanations in ML applications for physical sciences. Through three case studies in materials science and molecular property prediction, we examine how different models, explanation methods, levels of feature attribution, and stakeholder needs can result in varying interpretations of ML outputs. Our analysis underscores the importance of considering multiple perspectives when interpreting ML models in scientific contexts and highlights the critical need for scientists to maintain control over the interpretation process, balancing data-driven insights with domain expertise to meet specific scientific needs. By fostering a comprehensive understanding of these inconsistencies, we aim to contribute to the responsible integration of eXplainable Artificial Intelligence (XAI) into physical sciences and improve the trustworthiness of ML applications in scientific discovery.