Thunder-KoNUBench: A Corpus-Aligned Benchmark for Korean Negation Understanding
This addresses a gap in evaluating LLMs for Korean negation, which is important for NLP researchers and developers working on Korean language applications, though it is incremental as it adapts existing benchmark concepts to a specific language.
The authors tackled the lack of benchmarks for evaluating negation understanding in Korean by introducing Thunder-KoNUBench, a corpus-aligned benchmark, and found that fine-tuning on it improved negation understanding and broader contextual comprehension in Korean for 47 evaluated LLMs.
Although negation is known to challenge large language models (LLMs), benchmarks for evaluating negation understanding, especially in Korean, are scarce. We conduct a corpus-based analysis of Korean negation and show that LLM performance degrades under negation. We then introduce Thunder-KoNUBench, a sentence-level benchmark that reflects the empirical distribution of Korean negation phenomena. Evaluating 47 LLMs, we analyze the effects of model size and instruction tuning, and show that fine-tuning on Thunder-KoNUBench improves negation understanding and broader contextual comprehension in Korean.