Can LLM Assist in the Evaluation of the Quality of Machine Learning Explanations?
This work addresses the need for effective evaluation methods in explainable machine learning, though it is incremental as it builds on existing LLM-as-a-Judge approaches.
The paper tackles the problem of evaluating the quality of machine learning explanations by proposing a workflow that integrates LLM-based and human judges, finding that LLMs effectively assess explanations using subjective metrics but are not yet ready to replace human judges.
EXplainable machine learning (XML) has recently emerged to address the mystery mechanisms of machine learning (ML) systems by interpreting their 'black box' results. Despite the development of various explanation methods, determining the most suitable XML method for specific ML contexts remains unclear, highlighting the need for effective evaluation of explanations. The evaluating capabilities of the Transformer-based large language model (LLM) present an opportunity to adopt LLM-as-a-Judge for assessing explanations. In this paper, we propose a workflow that integrates both LLM-based and human judges for evaluating explanations. We examine how LLM-based judges evaluate the quality of various explanation methods and compare their evaluation capabilities to those of human judges within an iris classification scenario, employing both subjective and objective metrics. We conclude that while LLM-based judges effectively assess the quality of explanations using subjective metrics, they are not yet sufficiently developed to replace human judges in this role.