Use Sparse Autoencoders to Discover Unknown Concepts, Not to Act on Known Concepts
This clarifies the utility of sparse autoencoders for researchers in machine learning interpretability and domain sciences, though it is incremental as it refines existing understanding.
The paper argues that sparse autoencoders are ineffective for acting on known concepts but powerful for discovering unknown concepts, reconciling conflicting narratives and suggesting applications in interpretability and social sciences.
While sparse autoencoders (SAEs) have generated significant excitement, a series of negative results have added to skepticism about their usefulness. Here, we establish a conceptual distinction that reconciles competing narratives surrounding SAEs. We argue that while SAEs may be less effective for acting on known concepts, SAEs are powerful tools for discovering unknown concepts. This distinction cleanly separates existing negative and positive results, and suggests several classes of SAE applications. Specifically, we outline use cases for SAEs in (i) ML interpretability, explainability, fairness, auditing, and safety, and (ii) social and health sciences.