Interaction-Aware 4D Gaussian Splatting for Dynamic Hand-Object Interaction Reconstruction
This addresses the challenge of modeling hand-object interactions with mutual occlusion and motion for applications in VR/AR and robotics, representing an incremental improvement over existing dynamic 3D Gaussian splatting methods.
This paper tackles the problem of reconstructing dynamic hand-object interactions without object priors by introducing interaction-aware 4D Gaussian splatting with optimizable parameters and progressive optimization. The method achieves state-of-the-art performance in reconstructing these complex scenes.
This paper focuses on a challenging setting of simultaneously modeling geometry and appearance of hand-object interaction scenes without any object priors. We follow the trend of dynamic 3D Gaussian Splatting based methods, and address several significant challenges. To model complex hand-object interaction with mutual occlusion and edge blur, we present interaction-aware hand-object Gaussians with newly introduced optimizable parameters aiming to adopt piecewise linear hypothesis for clearer structural representation. Moreover, considering the complementarity and tightness of hand shape and object shape during interaction dynamics, we incorporate hand information into object deformation field, constructing interaction-aware dynamic fields to model flexible motions. To further address difficulties in the optimization process, we propose a progressive strategy that handles dynamic regions and static background step by step. Correspondingly, explicit regularizations are designed to stabilize the hand-object representations for smooth motion transition, physical interaction reality, and coherent lighting. Experiments show that our approach surpasses existing dynamic 3D-GS-based methods and achieves state-of-the-art performance in reconstructing dynamic hand-object interaction.