Atomic Literary Styling: Mechanistic Manipulation of Prose Generation in Neural Language Models
This work addresses a critical gap in mechanistic interpretability for AI alignment, revealing that observational correlations in neural networks do not imply causal necessity, which is an incremental but important insight for the field.
The researchers tackled the problem of understanding literary style in neural language models by analyzing GPT-2 neurons, finding that ablating discriminative neurons paradoxically improved generated prose quality by 25.7% in style metrics.
We present a mechanistic analysis of literary style in GPT-2, identifying individual neurons that discriminate between exemplary prose and rigid AI-generated text. Using Herman Melville's Bartleby, the Scrivener as a corpus, we extract activation patterns from 355 million parameters across 32,768 neurons in late layers. We find 27,122 statistically significant discriminative neurons ($p < 0.05$), with effect sizes up to $|d| = 1.4$. Through systematic ablation studies, we discover a paradoxical result: while these neurons correlate with literary text during analysis, removing them often improves rather than degrades generated prose quality. Specifically, ablating 50 high-discriminating neurons yields a 25.7% improvement in literary style metrics. This demonstrates a critical gap between observational correlation and causal necessity in neural networks. Our findings challenge the assumption that neurons which activate on desirable inputs will produce those outputs during generation, with implications for mechanistic interpretability research and AI alignment.