Uncovering Branch specialization in InceptionV1 using k sparse autoencoders
This work addresses the enigma of branch specialization in InceptionV1 for researchers in interpretable machine learning, but it appears incremental as it builds on prior sparse autoencoder methods.
The paper tackled the problem of understanding branch specialization in later layers of InceptionV1, showing examples of this phenomenon across specific branches and layers, with evidence that similar features are consistently localized in branches of the same convolution size.
Sparse Autoencoders (SAEs) have shown to find interpretable features in neural networks from polysemantic neurons caused by superposition. Previous work has shown SAEs are an effective tool to extract interpretable features from the early layers of InceptionV1. Since then, there have been many improvements to SAEs but branch specialization is still an enigma in the later layers of InceptionV1. We show various examples of branch specialization occuring in each layer of the mixed4a-4e branch, in the 5x5 branch and in one 1x1 branch. We also provide evidence to claim that branch specialization seems to be consistent across layers, similar features across the model will be localized in the same convolution size branches in their respective layer.