Shortcut Learning in In-Context Learning: A Survey
It addresses the issue of non-robust generalization in LLMs for researchers, but is incremental as it synthesizes existing work rather than presenting new findings.
This survey tackles the problem of shortcut learning in in-context learning for large language models, reviewing types, causes, benchmarks, and mitigation strategies, and outlines unresolved issues and future research directions.
Shortcut learning refers to the phenomenon where models employ simple, non-robust decision rules in practical tasks, which hinders their generalization and robustness. With the rapid development of large language models (LLMs) in recent years, an increasing number of studies have shown the impact of shortcut learning on LLMs. This paper provides a novel perspective to review relevant research on shortcut learning in In-Context Learning (ICL). It conducts a detailed exploration of the types of shortcuts in ICL tasks, their causes, available benchmarks, and strategies for mitigating shortcuts. Based on corresponding observations, it summarizes the unresolved issues in existing research and attempts to outline the future research landscape of shortcut learning.