Cycle Consistency in Video Object-Centric Learning
For researchers in self-supervised video object-centric learning, this work addresses a fundamental limitation of applying cycle consistency to latent slot spaces, offering a principled solution that avoids feature collapse.
The paper tackles the problem of applying cycle consistency to object-centric learning (OCL) in videos, where naive explicit cycle consistency causes feature collapse due to stochastic slot representations. They propose Implicit Cycle Consistency (ICC), which enforces consistency in the reconstruction manifold instead, outperforming explicit cycle consistency baselines on complex benchmarks.
Self-supervised video Object-Centric Learning (OCL) aims to discover distinct objects and associate them across time, whereas self-supervised Multi-Object Tracking (MOT) focuses on associating pre-defined object detections or segmentations. Although well-established in MOT, Cycle Consistency (CC) cannot naively or explicitly apply to the latent slot space of OCL. Unlike the deterministic and ideal object representations in MOT, OCL slots are inherently stochastic and ambiguous due to non-unique scene decompositions. Enforcing explicit cycle consistency (ECC) on slots imposes rigid mean seeking. This severely penalizes the model for exploring alternative but equally valid decompositions, thereby driving towards feature collapse. To resolve this dilemma, we propose \textit{Implicit Cycle Consistency (ICC)}, which shifts the cycle-consistency constraint from the restrictive slot space to the continuous reconstruction manifold, encouraging slots to reach a soft consensus on collectively interpreting the visual scene rather than forcing rigid point-to-point feature alignment. Extensive experiments on complex video OCL benchmarks demonstrate that ICC avoids feature collapse and outperforms ECC baselines. Our source code, model checkpoints and training logs are provided on https://github.com/Genera1Z/ICC.