Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Video
This work addresses the problem of limited tooling for researchers in vision mechanistic interpretability, enabling new research directions and lowering barriers to entry in this emerging field, though it is incremental as it builds on existing interpretability concepts from language models.
The authors tackled the lack of accessible frameworks and pre-trained models for mechanistic interpretability in vision and video by developing Prisma, an open-source toolkit that provides a unified framework for accessing over 75 vision and video transformers, training tools, and pre-trained weights, resulting in findings such as vision SAEs having lower sparsity than language SAEs and sometimes decreasing model loss.
Robust tooling and publicly available pre-trained models have helped drive recent advances in mechanistic interpretability for language models. However, similar progress in vision mechanistic interpretability has been hindered by the lack of accessible frameworks and pre-trained weights. We present Prisma (Access the codebase here: https://github.com/Prisma-Multimodal/ViT-Prisma), an open-source framework designed to accelerate vision mechanistic interpretability research, providing a unified toolkit for accessing 75+ vision and video transformers; support for sparse autoencoder (SAE), transcoder, and crosscoder training; a suite of 80+ pre-trained SAE weights; activation caching, circuit analysis tools, and visualization tools; and educational resources. Our analysis reveals surprising findings, including that effective vision SAEs can exhibit substantially lower sparsity patterns than language SAEs, and that in some instances, SAE reconstructions can decrease model loss. Prisma enables new research directions for understanding vision model internals while lowering barriers to entry in this emerging field.