Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization
This work addresses video understanding for researchers and practitioners by improving self-supervised learning, though it appears incremental as it builds on existing contrastive learning methods.
The paper tackled the problem of self-supervised video representation learning by addressing the neglect of lower-level features and temporal relationships, proposing a multi-level feature optimization framework that improved generalization and temporal modeling, with experiments showing significant enhancement in representation ability for video understanding.
The crux of self-supervised video representation learning is to build general features from unlabeled videos. However, most recent works have mainly focused on high-level semantics and neglected lower-level representations and their temporal relationship which are crucial for general video understanding. To address these challenges, this paper proposes a multi-level feature optimization framework to improve the generalization and temporal modeling ability of learned video representations. Concretely, high-level features obtained from naive and prototypical contrastive learning are utilized to build distribution graphs, guiding the process of low-level and mid-level feature learning. We also devise a simple temporal modeling module from multi-level features to enhance motion pattern learning. Experiments demonstrate that multi-level feature optimization with the graph constraint and temporal modeling can greatly improve the representation ability in video understanding. Code is available at https://github.com/shvdiwnkozbw/Video-Representation-via-Multi-level-Optimization.