Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
This addresses a critical flaw in interpretable AI for researchers and practitioners, though it is incremental as it builds on existing rationalization frameworks.
The study identified that cooperative self-rationalization frameworks can introduce spurious correlations between selected rationales and labels, even in clean datasets, and proposed a method to prevent this, achieving significant performance improvements over recent rationalization methods and comparable results to a large language model.
This study investigates the self-rationalization framework constructed with a cooperative game, where a generator initially extracts the most informative segment from raw input, and a subsequent predictor utilizes the selected subset for its input. The generator and predictor are trained collaboratively to maximize prediction accuracy. In this paper, we first uncover a potential caveat: such a cooperative game could unintentionally introduce a sampling bias during rationale extraction. Specifically, the generator might inadvertently create an incorrect correlation between the selected rationale candidate and the label, even when they are semantically unrelated in the original dataset. Subsequently, we elucidate the origins of this bias using both detailed theoretical analysis and empirical evidence. Our findings suggest a direction for inspecting these correlations through attacks, based on which we further introduce an instruction to prevent the predictor from learning the correlations. Through experiments on six text classification datasets and two graph classification datasets using three network architectures (GRUs, BERT, and GCN), we show that our method not only significantly outperforms recent rationalization methods, but also achieves comparable or even better results than a representative LLM (llama3.1-8b-instruct).