Binding Dynamics in Rotating Features
This work addresses a specific bottleneck in unsupervised object-centric representation learning for machine learning, but it is incremental as it replaces an existing mechanism without improving overall performance.
The paper tackles the problem of understanding the binding mechanism in Rotating Features, which is crucial for learning object-centric representations, by proposing an alternative cosine binding mechanism that achieves equivalent performance.
In human cognition, the binding problem describes the open question of how the brain flexibly integrates diverse information into cohesive object representations. Analogously, in machine learning, there is a pursuit for models capable of strong generalization and reasoning by learning object-centric representations in an unsupervised manner. Drawing from neuroscientific theories, Rotating Features learn such representations by introducing vector-valued features that encapsulate object characteristics in their magnitudes and object affiliation in their orientations. The "$χ$-binding" mechanism, embedded in every layer of the architecture, has been shown to be crucial, but remains poorly understood. In this paper, we propose an alternative "cosine binding" mechanism, which explicitly computes the alignment between features and adjusts weights accordingly, and we show that it achieves equivalent performance. This allows us to draw direct connections to self-attention and biological neural processes, and to shed light on the fundamental dynamics for object-centric representations to emerge in Rotating Features.