From a Bird's Eye View to See: Joint Camera and Subject Registration without the Camera Calibration
This addresses a challenging multi-view registration problem for computer vision applications like surveillance or autonomous systems, but it appears incremental as it builds on existing view transformation and geometric methods.
The paper tackles the problem of jointly registering multiple cameras and subjects in a bird's eye view without prior camera calibration, using only RGB images from first-person views, and achieves remarkable effectiveness in localizing and orienting both cameras and pedestrians in a unified plane.
We tackle a new problem of multi-view camera and subject registration in the bird's eye view (BEV) without pre-given camera calibration. This is a very challenging problem since its only input is several RGB images from different first-person views (FPVs) for a multi-person scene, without the BEV image and the calibration of the FPVs, while the output is a unified plane with the localization and orientation of both the subjects and cameras in a BEV. We propose an end-to-end framework solving this problem, whose main idea can be divided into following parts: i) creating a view-transform subject detection module to transform the FPV to a virtual BEV including localization and orientation of each pedestrian, ii) deriving a geometric transformation based method to estimate camera localization and view direction, i.e., the camera registration in a unified BEV, iii) making use of spatial and appearance information to aggregate the subjects into the unified BEV. We collect a new large-scale synthetic dataset with rich annotations for evaluation. The experimental results show the remarkable effectiveness of our proposed method.