AdaGaR: Adaptive Gabor Representation for Dynamic Scene Reconstruction
This work solves the problem of capturing high-frequency details and smooth motion in dynamic scene reconstruction for computer vision applications, representing an incremental improvement over existing methods.
The paper tackles dynamic 3D scene reconstruction from monocular videos by proposing AdaGaR, which addresses frequency adaptivity and temporal continuity, achieving state-of-the-art results with PSNR 35.49, SSIM 0.9433, and LPIPS 0.0723 on Tap-Vid DAVIS.
Reconstructing dynamic 3D scenes from monocular videos requires simultaneously capturing high-frequency appearance details and temporally continuous motion. Existing methods using single Gaussian primitives are limited by their low-pass filtering nature, while standard Gabor functions introduce energy instability. Moreover, lack of temporal continuity constraints often leads to motion artifacts during interpolation. We propose AdaGaR, a unified framework addressing both frequency adaptivity and temporal continuity in explicit dynamic scene modeling. We introduce Adaptive Gabor Representation, extending Gaussians through learnable frequency weights and adaptive energy compensation to balance detail capture and stability. For temporal continuity, we employ Cubic Hermite Splines with Temporal Curvature Regularization to ensure smooth motion evolution. An Adaptive Initialization mechanism combining depth estimation, point tracking, and foreground masks establishes stable point cloud distributions in early training. Experiments on Tap-Vid DAVIS demonstrate state-of-the-art performance (PSNR 35.49, SSIM 0.9433, LPIPS 0.0723) and strong generalization across frame interpolation, depth consistency, video editing, and stereo view synthesis. Project page: https://jiewenchan.github.io/AdaGaR/