Neuro-Symbolic Reasoning Shortcuts: Mitigation Strategies and their Limitations
This addresses a critical issue for researchers and practitioners in neuro-symbolic AI, though it is incremental as it builds on prior findings about reasoning shortcuts.
The paper tackles the problem of reasoning shortcuts in neuro-symbolic predictors, which lead to poor out-of-distribution performance and compromised interpretability, by establishing a formal link between these shortcuts and loss function optima and identifying their causes.
Neuro-symbolic predictors learn a mapping from sub-symbolic inputs to higher-level concepts and then carry out (probabilistic) logical inference on this intermediate representation. This setup offers clear advantages in terms of consistency to symbolic prior knowledge, and is often believed to provide interpretability benefits in that - by virtue of complying with the knowledge - the learned concepts can be better understood by human stakeholders. However, it was recently shown that this setup is affected by reasoning shortcuts whereby predictions attain high accuracy by leveraging concepts with unintended semantics, yielding poor out-of-distribution performance and compromising interpretability. In this short paper, we establish a formal link between reasoning shortcuts and the optima of the loss function, and identify situations in which reasoning shortcuts can arise. Based on this, we discuss limitations of natural mitigation strategies such as reconstruction and concept supervision.