SteerPose: Simultaneous Extrinsic Camera Calibration and Matching from Articulation
This addresses the challenge of extrinsic camera calibration and correspondence matching in multi-camera setups for computer vision applications, offering a novel approach that is incremental in combining neural networks with geometric consistency.
The paper tackles the problem of using freely moving humans or animals as calibration targets for multi-camera systems while estimating correspondences across views, proposing SteerPose, a neural network that rotates 2D poses into other views, with experimental results on diverse in-the-wild datasets validating its effectiveness and robustness.
Can freely moving humans or animals themselves serve as calibration targets for multi-camera systems while simultaneously estimating their correspondences across views? We humans can solve this problem by mentally rotating the observed 2D poses and aligning them with those in the target views. Inspired by this cognitive ability, we propose SteerPose, a neural network that performs this rotation of 2D poses into another view. By integrating differentiable matching, SteerPose simultaneously performs extrinsic camera calibration and correspondence search within a single unified framework. We also introduce a novel geometric consistency loss that explicitly ensures that the estimated rotation and correspondences result in a valid translation estimation. Experimental results on diverse in-the-wild datasets of humans and animals validate the effectiveness and robustness of the proposed method. Furthermore, we demonstrate that our method can reconstruct the 3D poses of novel animals in multi-camera setups by leveraging off-the-shelf 2D pose estimators and our class-agnostic model.