Human from Blur: Human Pose Tracking from Blurry Images
This addresses the challenge of human pose tracking in blurry images, which is important for applications like sports analysis or surveillance, and it is novel as the first method to tackle this specific problem.
The paper tackles the problem of estimating 3D human poses from blurry images by modeling the blurring process with a 3D human model and differentiable rendering, and it outperforms other methods on significantly blurry inputs.
We propose a method to estimate 3D human poses from substantially blurred images. The key idea is to tackle the inverse problem of image deblurring by modeling the forward problem with a 3D human model, a texture map, and a sequence of poses to describe human motion. The blurring process is then modeled by a temporal image aggregation step. Using a differentiable renderer, we can solve the inverse problem by backpropagating the pixel-wise reprojection error to recover the best human motion representation that explains a single or multiple input images. Since the image reconstruction loss alone is insufficient, we present additional regularization terms. To the best of our knowledge, we present the first method to tackle this problem. Our method consistently outperforms other methods on significantly blurry inputs since they lack one or multiple key functionalities that our method unifies, i.e. image deblurring with sub-frame accuracy and explicit 3D modeling of non-rigid human motion.