Framework for Machine Evaluation of Reasoning Completeness in Large Language Models For Classification Tasks
This addresses the need for transparent AI in sensitive domains by providing a quantitative method to assess reasoning completeness in LLMs, though it is incremental as it builds on existing evaluation techniques.
The paper tackles the problem of evaluating whether LLM-generated explanations faithfully capture predictive signals by introducing RACE, a framework that compares rationales against interpretable feature importance scores; results show correct predictions have higher coverage of supporting features while incorrect ones have elevated contradicting features, with edit-distance matching revealing paraphrastic overlaps.
The growing adoption of machine learning (ML) in sensitive domains has heightened the demand for transparent and interpretable artificial intelligence. Large Language Models (LLMs) are increasingly capable of producing natural language explanations, yet it remains unclear whether these rationales faithfully capture the predictive signals that underlie decisions. This paper introduces RACE-Reasoning Alignment for Completeness of Explanations, a systematic framework to evaluate the alignment between LLM-generated explanations and interpretable feature importance scores derived from a logistic regression baseline. We analyze four widely used text classification datasets-WIKI ONTOLOGY, AG NEWS, IMDB, and GOEMOTIONS-and compare LLM rationales against top-ranked supporting and contradicting lexical features. To capture alignment at multiple levels of granularity, RACE implements token-aware, exact string, and edit-distance matching techniques. Empirical results reveal a consistent asymmetry: correct predictions exhibit higher coverage of supporting features, while incorrect predictions are associated with elevated coverage of contradicting features. Edit-distance matching further uncovers paraphrastic overlaps, boosting coverage while preserving this asymmetry. These findings demonstrate that LLM rationales combine both surface-level and flexible evidence reuse, yet can also amplify misleading cues in error cases. RACE provides new insights into the faithfulness of LLM explanations and establishes a quantitative basis for evaluating reasoning completeness in neural language models.