Hansel: A Chinese Few-Shot and Zero-Shot Entity Linking Benchmark
This addresses the problem of popularity bias in entity linking for Chinese language researchers and practitioners, though it is incremental as it extends existing benchmark efforts to a new language.
The paper tackles the lack of non-English benchmarks for few-shot and zero-shot entity linking, particularly for tail and emerging entities, by introducing Hansel, a Chinese dataset, and shows that existing state-of-the-art systems perform poorly on it (R@1 of 36.6% on Few-Shot), while their baseline achieves R@1 of 46.2% on Few-Shot and 76.6% on Zero-Shot.
Modern Entity Linking (EL) systems entrench a popularity bias, yet there is no dataset focusing on tail and emerging entities in languages other than English. We present Hansel, a new benchmark in Chinese that fills the vacancy of non-English few-shot and zero-shot EL challenges. The test set of Hansel is human annotated and reviewed, created with a novel method for collecting zero-shot EL datasets. It covers 10K diverse documents in news, social media posts and other web articles, with Wikidata as its target Knowledge Base. We demonstrate that the existing state-of-the-art EL system performs poorly on Hansel (R@1 of 36.6% on Few-Shot). We then establish a strong baseline that scores a R@1 of 46.2% on Few-Shot and 76.6% on Zero-Shot on our dataset. We also show that our baseline achieves competitive results on TAC-KBP2015 Chinese Entity Linking task.