sim2art: Accurate Articulated Object Modeling from a Single Video using Synthetic Training Data Only
This provides a scalable solution for robotics and digital twin creation by enabling accurate modeling from casually recorded video.
The paper tackles the problem of modeling articulated objects from monocular video by jointly predicting part segmentation and joint parameters, achieving strong generalization to real-world objects using only synthetic training data.
Understanding articulated objects is a fundamental challenge in robotics and digital twin creation. To effectively model such objects, it is essential to recover both part segmentation and the underlying joint parameters. Despite the importance of this task, previous work has largely focused on setups like multi-view systems, object scanning, or static cameras. In this paper, we present the first data-driven approach that jointly predicts part segmentation and joint parameters from monocular video captured with a freely moving camera. Trained solely on synthetic data, our method demonstrates strong generalization to real-world objects, offering a scalable and practical solution for articulated object understanding. Our approach operates directly on casually recorded video, making it suitable for real-time applications in dynamic environments. Project webpage: https://aartykov.github.io/sim2art/