Rethinking Layer Relevance in Large Language Models Beyond Cosine Similarity
For researchers and practitioners working on LLM interpretability and pruning, this work highlights a fundamental flaw in a widely used metric and provides a more reliable alternative.
The paper shows that cosine similarity is a poor proxy for layer importance in LLMs, as it can be arbitrarily low for critical layers and correlates weakly with actual performance degradation. They propose using the actual accuracy drop from layer removal as a more robust metric.
Large language models (LLMs) have revolutionized natural language processing. Understanding their internal mechanisms is crucial for developing more interpretable and optimized architectures. Mechanistic interpretability has led to the development of various methods for assessing layer relevance, with cosine similarity being a widely used tool in the field. On this work, we demonstrate that cosine similarity is a poor proxy for the actual performance degradation caused by layer removal. Our theoretical analysis shows that a layer can exhibit an arbitrarily low cosine similarity score while still being crucial to the model's performance. On the other hand, empirical evidence from a range of LLMs confirms that the correlation between cosine similarity and actual performance degradation is often weak or moderate, leading to misleading interpretations of a transformer's internal mechanisms. We propose a more robust metric for assessing layer relevance: the actual drop in model accuracy resulting from the removal of a layer. Even though it is a computationally costly metric, this approach offers a more accurate picture of layer importance, allowing for more informed pruning strategies and lightweight models. Our findings have significant implications for the development of interpretable LLMs and highlight the need to move beyond cosine similarity in assessing layer relevance.