Beyond the Leaderboard: Understanding Performance Disparities in Large Language Models via Model Diffing
This provides a transparent framework for comparing LLMs, addressing the need for fine-grained insights beyond leaderboard metrics, though it is incremental as it applies an existing interpretability method to new models.
The paper tackled the problem of understanding performance disparities in large language models by using model diffing to analyze differences between Gemma-2-9b-it and a SimPO-enhanced variant, finding specific improvements such as a 151.7% increase in instruction-following and a 68.5% reduction in hallucination management.
As fine-tuning becomes the dominant paradigm for improving large language models (LLMs), understanding what changes during this process is increasingly important. Traditional benchmarking often fails to explain why one model outperforms another. In this work, we use model diffing, a mechanistic interpretability approach, to analyze the specific capability differences between Gemma-2-9b-it and a SimPO-enhanced variant. Using crosscoders, we identify and categorize latent representations that differentiate the two models. We find that SimPO acquired latent concepts predominantly enhance safety mechanisms (+32.8%), multilingual capabilities (+43.8%), and instruction-following (+151.7%), while its additional training also reduces emphasis on model self-reference (-44.1%) and hallucination management (-68.5%). Our analysis shows that model diffing can yield fine-grained insights beyond leaderboard metrics, attributing performance gaps to concrete mechanistic capabilities. This approach offers a transparent and targeted framework for comparing LLMs.