Few-Shot Neural Differentiable Simulator: Real-to-Sim Rigid-Contact Modeling
This work provides a method for improving the fidelity of physics simulations for robotic manipulation and control, particularly in scenarios involving complex contact dynamics, by reducing the need for extensive real-world data.
This paper addresses the challenge of accurately simulating complex contact dynamics for robotics by proposing a few-shot real-to-sim approach. It combines analytical simulators with GNNs, using a small amount of real-world data to calibrate the analytical simulator and generate large synthetic datasets. The method outperforms differentiable baselines in replicating real-world trajectories and improves the efficiency of policy learning.
Accurate physics simulation is essential for robotic learning and control, yet analytical simulators often fail to capture complex contact dynamics, while learning-based simulators typically require large amounts of costly real-world data. To bridge this gap, we propose a few-shot real-to-sim approach that combines the physical consistency of analytical formulations with the representational capacity of graph neural network (GNN)-based models. Using only a small amount of real-world data, our method calibrates analytical simulators to generate large-scale synthetic datasets that capture diverse contact interactions. On this foundation, we introduce a mesh-based GNN that implicitly models rigid-body forward dynamics and derive surrogate gradients for collision detection, achieving full differentiability. Experimental results demonstrate that our approach enables learning-based simulators to outperform differentiable baselines in replicating real-world trajectories. In addition, the differentiable design supports gradient-based optimization, which we validate through simulation-based policy learning in multi-object interaction scenarios. Extensive experiments show that our framework not only improves simulation fidelity with minimal supervision but also increases the efficiency of policy learning. Taken together, these findings suggest that differentiable simulation with few-shot real-world grounding provides a powerful direction for advancing future robotic manipulation and control.