A Fusion of Variational Distribution Priors and Saliency Map Replay for Continual 3D Reconstruction
This addresses the challenge of data acquisition and forgetting in learning-based 3D reconstruction for applications like robotics or AR/VR, though it is incremental as it builds on existing continual learning and 3D reconstruction techniques.
The paper tackles the problem of single-image 3D reconstruction in a continual learning setting, where models must retain performance on previously seen classes after training on new ones, and achieves competitive results compared to established methods.
Single-image 3D reconstruction is a research challenge focused on predicting 3D object shapes from single-view images. This task requires significant data acquisition to predict both visible and occluded portions of the shape. Furthermore, learning-based methods face the difficulty of creating a comprehensive training dataset for all possible classes. To this end, we propose a continual learning-based 3D reconstruction method where our goal is to design a model using Variational Priors that can still reconstruct the previously seen classes reasonably even after training on new classes. Variational Priors represent abstract shapes and combat forgetting, whereas saliency maps preserve object attributes with less memory usage. This is vital due to resource constraints in storing extensive training data. Additionally, we introduce saliency map-based experience replay to capture global and distinct object features. Thorough experiments show competitive results compared to established methods, both quantitatively and qualitatively.