Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos
This addresses the challenge of creating realistic animal avatars for applications like animation or virtual reality, though it is incremental as it builds on existing parametric models.
The paper tackles the problem of reconstructing animatable 3D dog avatars from monocular videos by improving template-based shape fitting with Continuous Surface Embeddings and modeling appearance with an implicit duplex-mesh texture, demonstrating superior results on CoP3D and APTv2 datasets compared to existing methods.
We present a method to build animatable dog avatars from monocular videos. This is challenging as animals display a range of (unpredictable) non-rigid movements and have a variety of appearance details (e.g., fur, spots, tails). We develop an approach that links the video frames via a 4D solution that jointly solves for animal's pose variation, and its appearance (in a canonical pose). To this end, we significantly improve the quality of template-based shape fitting by endowing the SMAL parametric model with Continuous Surface Embeddings, which brings image-to-mesh reprojection constaints that are denser, and thus stronger, than the previously used sparse semantic keypoint correspondences. To model appearance, we propose an implicit duplex-mesh texture that is defined in the canonical pose, but can be deformed using SMAL pose coefficients and later rendered to enforce a photometric compatibility with the input video frames. On the challenging CoP3D and APTv2 datasets, we demonstrate superior results (both in terms of pose estimates and predicted appearance) to existing template-free (RAC) and template-based approaches (BARC, BITE).