Patch-based Representation and Learning for Efficient Deformation Modeling
This work addresses the problem of slow deformation modeling in computer vision and graphics, offering incremental improvements in speed and generalization for tasks like shape-from-template and garment draping.
The paper tackles efficient deformation modeling for surfaces by introducing PolyFit, a patch-based representation learned from analytic functions and real data, which enables faster shape-from-template and garment draping with competitive accuracy and up to an order-of-magnitude speed improvements.
In this paper, we present a patch-based representation of surfaces, PolyFit, which is obtained by fitting jet functions locally on surface patches. Such a representation can be learned efficiently in a supervised fashion from both analytic functions and real data. Once learned, it can be generalized to various types of surfaces. Using PolyFit, the surfaces can be efficiently deformed by updating a compact set of jet coefficients rather than optimizing per-vertex degrees of freedom for many downstream tasks in computer vision and graphics. We demonstrate the capabilities of our proposed methodologies with two applications: 1) Shape-from-template (SfT): where the goal is to deform the input 3D template of an object as seen in image/video. Using PolyFit, we adopt test-time optimization that delivers competitive accuracy while being markedly faster than offline physics-based solvers, and outperforms recent physics-guided neural simulators in accuracy at modest additional runtime. 2) Garment draping. We train a self-supervised, mesh- and garment-agnostic model that generalizes across resolutions and garment types, delivering up to an order-of-magnitude faster inference than strong baselines.