FINCH: Locally Visualizing Higher-Order Feature Interactions in Black Box Models
This addresses the need for better interpretability in black-box models for users in industries relying on AI decision-making, though it is incremental as it builds on existing explanation methods by focusing on interactions.
The paper tackled the problem of visualizing higher-order feature interactions in black-box AI models for local explanations, resulting in a visual analytics tool called FINCH that enables intuitive, human-centered visualizations and was validated through case studies and a human study with machine learning experts.
In an era where black-box AI models are integral to decision-making across industries, robust methods for explaining these models are more critical than ever. While these models leverage complex feature interplay for accurate predictions, most explanation methods only assign relevance to individual features. There is a research gap in methods that effectively illustrate interactions between features, especially in visualizing higher-order interactions involving multiple features, which challenge conventional representation methods. To address this challenge in local explanations focused on individual instances, we employ a visual, subset-based approach to reveal relevant feature interactions. Our visual analytics tool FINCH uses coloring and highlighting techniques to create intuitive, human-centered visualizations, and provides additional views that enable users to calibrate their trust in the model and explanations. We demonstrate FINCH in multiple case studies, demonstrating its generalizability, and conducted an extensive human study with machine learning experts to highlight its helpfulness and usability. With this approach, FINCH allows users to visualize feature interactions involving any number of features locally.