Human Evaluation of Interpretability: The Case of AI-Generated Music Knowledge
This work addresses the need for better interpretability evaluation in AI knowledge discovery for researchers in AI and HCI, though it is incremental as it builds on existing interpretability research.
The paper tackles the problem of evaluating the interpretability of AI-generated knowledge in the arts and humanities, specifically focusing on music theory rules, by developing an experimental procedure to collect human interpretations, revealing both possibilities and challenges in decoding AI messages.
Interpretability of machine learning models has gained more and more attention among researchers in the artificial intelligence (AI) and human-computer interaction (HCI) communities. Most existing work focuses on decision making, whereas we consider knowledge discovery. In particular, we focus on evaluating AI-discovered knowledge/rules in the arts and humanities. From a specific scenario, we present an experimental procedure to collect and assess human-generated verbal interpretations of AI-generated music theory/rules rendered as sophisticated symbolic/numeric objects. Our goal is to reveal both the possibilities and the challenges in such a process of decoding expressive messages from AI sources. We treat this as a first step towards 1) better design of AI representations that are human interpretable and 2) a general methodology to evaluate interpretability of AI-discovered knowledge representations.