Rotary Offset Features in Large Language Models
This addresses the understanding of positional encoding patterns in LLMs, but it is incremental as it focuses on analyzing existing features rather than introducing new methods.
The paper analyzed the emergence of rotary offset features in transformer-based LLMs using Rotary Positional Encodings, revealing that these features consistently appear as large activations across layers and architectures, with derived bounds predicting their frequencies and angles.
Transformer-based Large Language Models (LLMs) rely on positional encodings to provide sequence position information to their attention mechanism. Rotary Positional Encodings (RoPE), which encode relative position by rotating queries and keys, have become widely used in modern LLMs. We study the features and patterns that emerge in queries and keys when using rotary embeddings and introduce the concept of rotary offset features. Our analysis reveals that these features, which frequently exhibit large activations and are often interpreted as outliers, arise consistently across layers, attention heads, and model architectures. We derive bounds predicting which rotary frequencies give rise to rotary offset features and the minimum angle between the query-key pairs for these features. We verify our predictions empirically across models of different sizes and architectures.