Suppressing Static Visual Cues via Normalizing Flows for Self-Supervised Video Representation Learning
This addresses the issue of biased video features for researchers in computer vision, offering an incremental improvement over existing self-supervised methods.
The paper tackles the problem of video representation learning being biased towards static visual cues by proposing a method to suppress static cues using normalizing flows, resulting in state-of-the-art performance on benchmarks when using only RGB data for pre-training.
Despite the great progress in video understanding made by deep convolutional neural networks, feature representation learned by existing methods may be biased to static visual cues. To address this issue, we propose a novel method to suppress static visual cues (SSVC) based on probabilistic analysis for self-supervised video representation learning. In our method, video frames are first encoded to obtain latent variables under standard normal distribution via normalizing flows. By modelling static factors in a video as a random variable, the conditional distribution of each latent variable becomes shifted and scaled normal. Then, the less-varying latent variables along time are selected as static cues and suppressed to generate motion-preserved videos. Finally, positive pairs are constructed by motion-preserved videos for contrastive learning to alleviate the problem of representation bias to static cues. The less-biased video representation can be better generalized to various downstream tasks. Extensive experiments on publicly available benchmarks demonstrate that the proposed method outperforms the state of the art when only single RGB modality is used for pre-training.