Crafting Large Language Models for Enhanced Interpretability
This addresses the need for transparency and clear explanations in language models for users and developers, representing a novel approach rather than an incremental improvement.
The paper tackles the problem of black-box interpretability in Large Language Models by introducing the Concept Bottleneck Large Language Model (CB-LLM), which achieves built-in interpretability while narrowing the performance gap with traditional models through an Automatic Concept Correction strategy.
We introduce the Concept Bottleneck Large Language Model (CB-LLM), a pioneering approach to creating inherently interpretable Large Language Models (LLMs). Unlike traditional black-box LLMs that rely on post-hoc interpretation methods with limited neuron function insights, CB-LLM sets a new standard with its built-in interpretability, scalability, and ability to provide clear, accurate explanations. This innovation not only advances transparency in language models but also enhances their effectiveness. Our unique Automatic Concept Correction (ACC) strategy successfully narrows the performance gap with conventional black-box LLMs, positioning CB-LLM as a model that combines the high accuracy of traditional LLMs with the added benefit of clear interpretability -- a feature markedly absent in existing LLMs.