I Walk the Line: Examining the Role of Gestalt Continuity in Object Binding for Vision Transformers
This work addresses the mechanism of object binding in vision transformers, providing insights into how these models achieve perceptual grouping, which is a key step toward flexible visual intelligence.
The paper investigates whether vision transformers use Gestalt continuity for object binding, finding that binding probes are sensitive to continuity across models, identifying specific attention heads that track continuity and contribute to binding representations.
Object binding is a foundational process in visual cognition, during which low-level perceptual features are joined into object representations. Binding has been considered a fundamental challenge for neural networks, and a major milestone on the way to artificial models with flexible visual intelligence. Recently, several investigations have demonstrated evidence that binding mechanisms emerge in pretrained vision models, enabling them to associate portions of an image that contain an object. The question remains: how are these models binding objects together? In this work, we investigate whether vision models rely on the principle of Gestalt continuity to perform object binding, over and above other principles like similarity and proximity. Using synthetic datasets, we demonstrate that binding probes are sensitive to continuity across a wide range of pretrained vision transformers. Next, we uncover particular attention heads that track continuity, and show that these heads generalize across datasets. Finally, we ablate these attention heads, and show that they often contribute to producing representations that encode object binding.