User-centric Subjective Leaderboard by Customizable Reward Modeling
This addresses the challenge for users in finding suitable LLMs for individual needs by offering a more practical, subjective evaluation method.
The paper tackles the problem of static benchmarks for LLM selection by introducing a User-Centric Subjective Leaderboard (USL) that provides dynamic, preference-driven rankings, achieving strong performance with a 4B-parameter Customizable Reward Model that surpasses GPT-4.1 and Gemini-2.5-pro.
Existing benchmarks for large language models (LLMs) predominantely focus on assessing their capabilities through verifiable tasks. Such objective and static benchmarks offer limited utility for practical LLM selection, making it difficult for users to find suitable models for their individual needs. To bridge this gap, we present the first User-Centric Subjective Leaderboard (USL), which provides a preference-driven, dynamic ranking of LLMs across diverse real-world scenarios. Our work is built upon a thorough investigation of real human preference data, involving more than 10K subjective queries. Our investigation reveals significant diversity and contradictions in human preferences, which limit the effectiveness of state-of-the-art reward models. To address this, we introduce Customizable Reward Models (CRMs). With only 4B parameters, our CRM surpasses the performance of leading models such as GPT-4.1 and Gemini-2.5-pro, showing exceptional generalization capabilities across new topics and criteria. The USL, powered by CRMs, exhibits strong negative correlations to contradictory preferences.