Estimating more camera poses for ego-centric videos is essential for VQ3D
This work addresses the challenge of 3D localization in egocentric videos for the VQ3D task, representing an incremental improvement over existing methods.
The paper tackles the problem of low query with pose ratio in VQ3D by designing a new pipeline for camera pose estimation in egocentric videos and optimizing the VQ3D framework, achieving a top-1 overall success rate of 25.8%, which is two times better than the baseline of 8.7%.
Visual queries 3D localization (VQ3D) is a task in the Ego4D Episodic Memory Benchmark. Given an egocentric video, the goal is to answer queries of the form "Where did I last see object X?", where the query object X is specified as a static image, and the answer should be a 3D displacement vector pointing to object X. However, current techniques use naive ways to estimate the camera poses of video frames, resulting in a low query with pose (QwP) ratio, thus a poor overall success rate. We design a new pipeline for the challenging egocentric video camera pose estimation problem in our work. Moreover, we revisit the current VQ3D framework and optimize it in terms of performance and efficiency. As a result, we get the top-1 overall success rate of 25.8% on VQ3D leaderboard, which is two times better than the 8.7% reported by the baseline.