In Search of Grandmother Cells: Tracing Interpretable Neurons in Tabular Representations
This addresses interpretability challenges in AI for researchers and practitioners, but it is incremental as it builds on existing ideas about grandmother cells.
The paper tackled the problem of opaque decision-making in foundation models by searching for interpretable neurons that respond to single concepts, finding that some neurons in TabPFN show moderate, statistically significant saliency and selectivity for high-level concepts.
Foundation models are powerful yet often opaque in their decision-making. A topic of continued interest in both neuroscience and artificial intelligence is whether some neurons behave like grandmother cells, i.e., neurons that are inherently interpretable because they exclusively respond to single concepts. In this work, we propose two information-theoretic measures that quantify the neuronal saliency and selectivity for single concepts. We apply these metrics to the representations of TabPFN, a tabular foundation model, and perform a simple search across neuron-concept pairs to find the most salient and selective pair. Our analysis provides the first evidence that some neurons in such models show moderate, statistically significant saliency and selectivity for high-level concepts. These findings suggest that interpretable neurons can emerge naturally and that they can, in some cases, be identified without resorting to more complex interpretability techniques.