Post-hoc Interpretability Illumination for Scientific Interaction Discovery
This addresses the need for better interpretability in decision-making applications, particularly for scientific discovery across diverse fields, though it appears incremental as it builds on existing interpretability methods.
The paper tackled the problem of limited capabilities and efficiency in existing interpretability tools by proposing Iterative Kings' Forests (iKF), a post-hoc method that uncovers complex multi-order interactions among variables, resulting in strong interpretive power demonstrated through extensive experiments.
Model interpretability and explainability have garnered substantial attention in recent years, particularly in decision-making applications. However, existing interpretability tools often fall short in delivering satisfactory performance due to limited capabilities or efficiency issues. To address these challenges, we propose a novel post-hoc method: Iterative Kings' Forests (iKF), designed to uncover complex multi-order interactions among variables. iKF iteratively selects the next most important variable, the "King", and constructs King's Forests by placing it at the root node of each tree to identify variables that interact with the "King". It then generates ranked short lists of important variables and interactions of varying orders. Additionally, iKF provides inference metrics to analyze the patterns of the selected interactions and classify them into one of three interaction types: Accompanied Interaction, Synergistic Interaction, and Hierarchical Interaction. Extensive experiments demonstrate the strong interpretive power of our proposed iKF, highlighting its great potential for explainable modeling and scientific discovery across diverse scientific fields.