Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects
This addresses the challenge of manipulating poses for new object categories in computer vision and graphics, offering an unsupervised approach that is not incremental but novel for this specific bottleneck.
The paper tackles the problem of learning the appearance and underlying joint structure of previously unseen articulated objects from multi-view video without supervision, enabling pose manipulation for applications like virtual reality and animation. The method models parts as 3D ellipsoids to identify joints and works on diverse structures such as quadrupeds, robots, and humans.
Rendering articulated objects while controlling their poses is critical to applications such as virtual reality or animation for movies. Manipulating the pose of an object, however, requires the understanding of its underlying structure, that is, its joints and how they interact with each other. Unfortunately, assuming the structure to be known, as existing methods do, precludes the ability to work on new object categories. We propose to learn both the appearance and the structure of previously unseen articulated objects by observing them move from multiple views, with no joints annotation supervision, or information about the structure. We observe that 3D points that are static relative to one another should belong to the same part, and that adjacent parts that move relative to each other must be connected by a joint. To leverage this insight, we model the object parts in 3D as ellipsoids, which allows us to identify joints. We combine this explicit representation with an implicit one that compensates for the approximation introduced. We show that our method works for different structures, from quadrupeds, to single-arm robots, to humans.