CLICKER: Attention-Based Cross-Lingual Commonsense Knowledge Transfer
This addresses the problem of performance gaps in commonsense reasoning for non-English speakers, though it appears incremental as it builds on existing multilingual pre-trained models.
The paper tackles the challenge of transferring commonsense knowledge from English to other languages in cross-lingual commonsense reasoning, proposing the CLICKER framework that improves performance for non-English languages by differentiating non-commonsense knowledge.
Recent advances in cross-lingual commonsense reasoning (CSR) are facilitated by the development of multilingual pre-trained models (mPTMs). While mPTMs show the potential to encode commonsense knowledge for different languages, transferring commonsense knowledge learned in large-scale English corpus to other languages is challenging. To address this problem, we propose the attention-based Cross-LIngual Commonsense Knowledge transfER (CLICKER) framework, which minimizes the performance gaps between English and non-English languages in commonsense question-answering tasks. CLICKER effectively improves commonsense reasoning for non-English languages by differentiating non-commonsense knowledge from commonsense knowledge. Experimental results on public benchmarks demonstrate that CLICKER achieves remarkable improvements in the cross-lingual CSR task for languages other than English.