Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
For interpretability researchers, this provides a principled framework to decompose neural representations into structured, low-dimensional concept manifolds, improving upon direction-based decomposition methods.
The paper introduces block-sparse featurizers (BSFs) that group neural network activation directions into blocks, capturing visual concepts as low-dimensional manifolds (2-4 dimensions). BSFs describe activations more compactly than direction-based methods, enabling discovery of novel manifolds (e.g., shadows, lighting) and interpretable control of diffusion model image generation.
What is the geometry of a visual percept? The most widely used protocols for decomposing neural network representations into interpretable parts treat concepts as isolated directions, yet recent work shows that concepts are often realized as geometric structures in low dimensional regions of activation space. We turn to the literature of Structured sparsity to close this gap, and show that block sparsity, which groups directions into blocks, is the prior matched to a generative model in which a representation is a sparse sum of low-dimensional manifolds: the modern, learned form of a classical idea in visual neuroscience, where a visual feature is carried by a coordinated group of neurons rather than a single tuned one. We implement three variants of block-sparse featurizers (BSFs) and, through a minimum-description-length analysis, show that all three describe activations more compactly than direction-based featurizers, with the recovered concepts typically two- to four-dimensional. We then use BSFs to (i) recontextualize prior work, showing that curve detectors in InceptionV1 actually read from a single continuous curve manifold, (ii) discover novel manifolds including shadows and lighting in DINOv3, and (iii) support interpretable control of image generation in diffusion models (SDXL) via manifold steering.