Punctuation and Predicates in Language Models
This work provides insights into the internal mechanisms of LLMs, aiding interpretability for researchers, but it is incremental as it builds on prior findings about punctuation and reasoning.
The paper investigates how punctuation tokens and reasoning rules like conditional statements are processed across layers in large language models (GPT-2, DeepSeek, Gemma), finding model-specific differences such as punctuation being necessary and sufficient in GPT-2 but not in others, and distinct processing for different reasoning components.
In this paper we explore where information is collected and how it is propagated throughout layers in large language models (LLMs). We begin by examining the surprising computational importance of punctuation tokens which previous work has identified as attention sinks and memory aids. Using intervention-based techniques, we evaluate the necessity and sufficiency (for preserving model performance) of punctuation tokens across layers in GPT-2, DeepSeek, and Gemma. Our results show stark model-specific differences: for GPT-2, punctuation is both necessary and sufficient in multiple layers, while this holds far less in DeepSeek and not at all in Gemma. Extending beyond punctuation, we ask whether LLMs process different components of input (e.g., subjects, adjectives, punctuation, full sentences) by forming early static summaries reused across the network, or if the model remains sensitive to changes in these components across layers. Extending beyond punctuation, we investigate whether different reasoning rules are processed differently by LLMs. In particular, through interchange intervention and layer-swapping experiments, we find that conditional statements (if, then), and universal quantification (for all) are processed very differently. Our findings offer new insight into the internal mechanisms of punctuation usage and reasoning in LLMs and have implications for interpretability.