Advances and Trends in the 3D Reconstruction of the Shape and Motion of Animals
It tackles the problem of non-intrusive 3D animal modeling for applications in biology, conservation, and entertainment, but is incremental as it reviews existing methods rather than proposing new ones.
This paper surveys deep learning-based techniques for 3D reconstruction of animal shape and motion from RGB images/videos, addressing the limitations of traditional intrusive methods like 3D scanners, and categorizes state-of-the-art methods while analyzing their performance and challenges.
Reconstructing the 3D geometry, pose, and motion of animals is a long-standing problem, which has a wide range of applications, from biology, livestock management, and animal conservation and welfare to content creation in digital entertainment and Virtual/Augmented Reality (VR/AR). Traditionally, 3D models of real animals are obtained using 3D scanners. These, however, are intrusive, often prohibitively expensive, and difficult to deploy in the natural environment of the animals. In recent years, we have seen a significant surge in deep learning-based techniques that enable the 3D reconstruction, in a non-intrusive manner, of the shape and motion of dynamic objects just from their RGB image and/or video observations. Several papers have explored their application and extension to various types of animals. This paper surveys the latest developments in this emerging and growing field of research. It categorizes and discusses the state-of-the-art methods based on their input modalities, the way the 3D geometry and motion of animals are represented, the type of reconstruction techniques they use, and the training mechanisms they adopt. It also analyzes the performance of some key methods, discusses their strengths and limitations, and identifies current challenges and directions for future research.