Pose-Aware Self-Supervised Learning with Viewpoint Trajectory Regularization
This work addresses the under-explored challenge of unsupervised feature learning for viewpoint reasoning, which is important for applications like autonomous driving, but it is incremental as it builds on existing self-supervised learning methods with a new regularization approach.
The paper tackles the problem of learning visual features that encode both object identity and pose from unlabeled images, introducing a viewpoint trajectory regularization loss and a new dataset of adjacent image triplets. The result is a representation that retains semantic classification accuracy while achieving emergent global pose awareness and better generalization to novel objects, as demonstrated in experiments.
Learning visual features from unlabeled images has proven successful for semantic categorization, often by mapping different $views$ of the same object to the same feature to achieve recognition invariance. However, visual recognition involves not only identifying $what$ an object is but also understanding $how$ it is presented. For example, seeing a car from the side versus head-on is crucial for deciding whether to stay put or jump out of the way. While unsupervised feature learning for downstream viewpoint reasoning is important, it remains under-explored, partly due to the lack of a standardized evaluation method and benchmarks. We introduce a new dataset of adjacent image triplets obtained from a viewpoint trajectory, without any semantic or pose labels. We benchmark both semantic classification and pose estimation accuracies on the same visual feature. Additionally, we propose a viewpoint trajectory regularization loss for learning features from unlabeled image triplets. Our experiments demonstrate that this approach helps develop a visual representation that encodes object identity and organizes objects by their poses, retaining semantic classification accuracy while achieving emergent global pose awareness and better generalization to novel objects. Our dataset and code are available at http://pwang.pw/trajSSL/.