NeuronScope: A Multi-Agent Framework for Explaining Polysemantic Neurons in Language Models
This addresses the challenge of neuron-level interpretation in LLMs for researchers, though it appears incremental as it builds on existing interpretation methods.
The paper tackles the problem of interpreting polysemantic neurons in large language models, where individual neurons respond to multiple concepts, by proposing NeuronScope, a multi-agent framework that deconstructs activations into semantic components and clusters them. The result shows that NeuronScope uncovers hidden polysemanticity and achieves significantly higher activation correlation compared to baselines.
Neuron-level interpretation in large language models (LLMs) is fundamentally challenged by widespread polysemanticity, where individual neurons respond to multiple distinct semantic concepts. Existing single-pass interpretation methods struggle to faithfully capture such multi-concept behavior. In this work, we propose NeuronScope, a multi-agent framework that reformulates neuron interpretation as an iterative, activation-guided process. NeuronScope explicitly deconstructs neuron activations into atomic semantic components, clusters them into distinct semantic modes, and iteratively refines each explanation using neuron activation feedback. Experiments demonstrate that NeuronScope uncovers hidden polysemanticity and produces explanations with significantly higher activation correlation compared to single-pass baselines.