Benchmark of Benchmarks: Unpacking Influence and Code Repository Quality in LLM Safety Benchmarks
This work addresses the problem of unclear benchmark prominence and quality for researchers in LLM safety, providing systematic insights to improve standards.
This paper conducted the first multi-dimensional evaluation of the influence and code quality of 31 LLM safety benchmarks, finding that benchmark papers do not have a significant advantage in academic influence over non-benchmark papers, and revealing misalignments where author prominence correlates with influence but not with code quality, with only 39% of repositories being ready-to-use.
The rapid growth of research in LLM safety makes it hard to track all advances. Benchmarks are therefore crucial for capturing key trends and enabling systematic comparisons. Yet, it remains unclear why certain benchmarks gain prominence, and no systematic assessment has been conducted on their academic influence or code quality. This paper fills this gap by presenting the first multi-dimensional evaluation of the influence (based on five metrics) and code quality (based on both automated and human assessment) on LLM safety benchmarks, analyzing 31 benchmarks and 382 non-benchmarks across prompt injection, jailbreak, and hallucination. We find that benchmark papers show no significant advantage in academic influence (e.g., citation count and density) over non-benchmark papers. We uncover a key misalignment: while author prominence correlates with paper influence, neither author prominence nor paper influence shows a significant correlation with code quality. Our results also indicate substantial room for improvement in code and supplementary materials: only 39% of repositories are ready-to-use, 16% include flawless installation guides, and a mere 6% address ethical considerations. Given that the work of prominent researchers tends to attract greater attention, they need to lead the effort in setting higher standards.