In-Context Explainers: Harnessing LLMs for Explaining Black Box Models
This addresses the need for interpretable AI by providing a novel method to explain complex predictive models, though it is an incremental advancement in applying LLMs to explanation tasks.
The paper tackled the problem of generating post hoc explanations for black box models by proposing In-Context Explainers, a framework that uses LLMs' in-context learning to explain predictions, and demonstrated that LLMs perform similarly to state-of-the-art explainers on real-world datasets.
Recent advancements in Large Language Models (LLMs) have demonstrated exceptional capabilities in complex tasks like machine translation, commonsense reasoning, and language understanding. One of the primary reasons for the adaptability of LLMs in such diverse tasks is their in-context learning (ICL) capability, which allows them to perform well on new tasks by simply using a few task samples in the prompt. Despite their effectiveness in enhancing the performance of LLMs on diverse language and tabular tasks, these methods have not been thoroughly explored for their potential to generate post hoc explanations. In this work, we carry out one of the first explorations to analyze the effectiveness of LLMs in explaining other complex predictive models using ICL. To this end, we propose a novel framework, In-Context Explainers, comprising of three novel approaches that exploit the ICL capabilities of LLMs to explain the predictions made by other predictive models. We conduct extensive analysis with these approaches on real-world tabular and text datasets and demonstrate that LLMs are capable of explaining other predictive models similar to state-of-the-art post hoc explainers, opening up promising avenues for future research into LLM-based post hoc explanations of complex predictive models.