MASIV: Toward Material-Agnostic System Identification from Videos
This addresses the limitation of existing methods that rely on predefined material priors, enabling broader application in robotics and simulation for handling diverse materials, though it is incremental by building on differentiable rendering and simulation approaches.
The paper tackles the problem of system identification from videos for unknown materials by introducing MASIV, a material-agnostic framework that uses learnable neural constitutive models and dense geometric guidance, achieving state-of-the-art performance in geometric accuracy, rendering quality, and generalization.
System identification from videos aims to recover object geometry and governing physical laws. Existing methods integrate differentiable rendering with simulation but rely on predefined material priors, limiting their ability to handle unknown ones. We introduce MASIV, the first vision-based framework for material-agnostic system identification. Unlike existing approaches that depend on hand-crafted constitutive laws, MASIV employs learnable neural constitutive models, inferring object dynamics without assuming a scene-specific material prior. However, the absence of full particle state information imposes unique challenges, leading to unstable optimization and physically implausible behaviors. To address this, we introduce dense geometric guidance by reconstructing continuum particle trajectories, providing temporally rich motion constraints beyond sparse visual cues. Comprehensive experiments show that MASIV achieves state-of-the-art performance in geometric accuracy, rendering quality, and generalization ability.