ANHALTEN: Cross-Lingual Transfer for German Token-Level Reference-Free Hallucination Detection
This work addresses a gap in NLP for German hallucination detection, enabling cross-lingual transfer with minimal annotation effort, though it is incremental as it extends existing English methods to a new language.
The paper tackles the lack of datasets for token-level reference-free hallucination detection in non-English languages by introducing ANHALTEN, a German dataset parallel to an English one, and shows that few-shot transfer is the most effective approach, achieving better detection with larger context length.
Research on token-level reference-free hallucination detection has predominantly focused on English, primarily due to the scarcity of robust datasets in other languages. This has hindered systematic investigations into the effectiveness of cross-lingual transfer for this important NLP application. To address this gap, we introduce ANHALTEN, a new evaluation dataset that extends the English hallucination detection dataset to German. To the best of our knowledge, this is the first work that explores cross-lingual transfer for token-level reference-free hallucination detection. ANHALTEN contains gold annotations in German that are parallel (i.e., directly comparable to the original English instances). We benchmark several prominent cross-lingual transfer approaches, demonstrating that larger context length leads to better hallucination detection in German, even without succeeding context. Importantly, we show that the sample-efficient few-shot transfer is the most effective approach in most setups. This highlights the practical benefits of minimal annotation effort in the target language for reference-free hallucination detection. Aiming to catalyze future research on cross-lingual token-level reference-free hallucination detection, we make ANHALTEN publicly available: https://github.com/janekh24/anhalten