CLSep 17, 2025
Characterizing Knowledge Graph Tasks in LLM Benchmarks Using Cognitive Complexity FrameworksSara Todorovikj, Lars-Peter Meyer, Michael Martin
Large Language Models (LLMs) are increasingly used for tasks involving Knowledge Graphs (KGs), whose evaluation typically focuses on accuracy and output correctness. We propose a complementary task characterization approach using three complexity frameworks from cognitive psychology. Applying this to the LLM-KG-Bench framework, we highlight value distributions, identify underrepresented demands and motivate richer interpretation and diversity for benchmark evaluation tasks.