Evaluating Latent Generative Paradigms for High-Fidelity 3D Shape Completion from a Single Depth Image
This work addresses the need for effective generative models in 3D shape completion, which is incremental as it compares existing methods on a specific task.
The paper tackled the problem of high-fidelity 3D shape completion from a single depth image by comparing Denoising Diffusion Probabilistic Models and Autoregressive Causal Transformers, finding that the diffusion model with continuous latents outperformed others and achieved state-of-the-art performance, while the autoregressive model matched or exceeded it in discrete latent spaces.
While generative models have seen significant adoption across a wide range of data modalities, including 3D data, a consensus on which model is best suited for which task has yet to be reached. Further, conditional information such as text and images to steer the generation process are frequently employed, whereas others, like partial 3D data, have not been thoroughly evaluated. In this work, we compare two of the most promising generative models--Denoising Diffusion Probabilistic Models and Autoregressive Causal Transformers--which we adapt for the tasks of generative shape modeling and completion. We conduct a thorough quantitative evaluation and comparison of both tasks, including a baseline discriminative model and an extensive ablation study. Our results show that (1) the diffusion model with continuous latents outperforms both the discriminative model and the autoregressive approach and delivers state-of-the-art performance on multi-modal shape completion from a single, noisy depth image under realistic conditions and (2) when compared on the same discrete latent space, the autoregressive model can match or exceed diffusion performance on these tasks.