CA$^2$T-Net: Category-Agnostic 3D Articulation Transfer from Single Image
This addresses the challenge of automating 3D animation and motion inference from images for arbitrary articulated objects, which is incremental as it builds on neural network approaches but extends to category-agnostic transfer.
The paper tackles the problem of transferring motion from a single image of an articulated object to a rest-state 3D model, achieving category-agnostic articulation transfer that works with arbitrary object topologies and can animate meshes or infer motion from real images.
We present a neural network approach to transfer the motion from a single image of an articulated object to a rest-state (i.e., unarticulated) 3D model. Our network learns to predict the object's pose, part segmentation, and corresponding motion parameters to reproduce the articulation shown in the input image. The network is composed of three distinct branches that take a shared joint image-shape embedding and is trained end-to-end. Unlike previous methods, our approach is independent of the topology of the object and can work with objects from arbitrary categories. Our method, trained with only synthetic data, can be used to automatically animate a mesh, infer motion from real images, and transfer articulation to functionally similar but geometrically distinct 3D models at test time.