DmifNet:3D Shape Reconstruction Based on Dynamic Multi-Branch Information Fusion
This work is significant for researchers and practitioners in computer vision and 3D graphics who need to reconstruct detailed 3D models from 2D images, especially for objects with intricate geometries.
This paper addresses the challenge of 3D object reconstruction from single-view images, particularly for complex topologies with rich edge and corner details. The proposed DmifNet recovers high-fidelity 3D shapes and aims to mitigate domain adaptation issues when moving from synthetic to real data.
3D object reconstruction from a single-view image is a long-standing challenging problem. Previous work was difficult to accurately reconstruct 3D shapes with a complex topology which has rich details at the edges and corners. Moreover, previous works used synthetic data to train their network, but domain adaptation problems occurred when tested on real data. In this paper, we propose a Dynamic Multi-branch Information Fusion Network (DmifNet) which can recover a high-fidelity 3D shape of arbitrary topology from a 2D image. Specifically, we design several side branches from the intermediate layers to make the network produce more diverse representations to improve the generalization ability of network. In addition, we utilize DoG (Difference of Gaussians) to extract edge geometry and corners information from input images. Then, we use a separate side branch network to process the extracted data to better capture edge geometry and corners feature information. Finally, we dynamically fuse the information of all branches to gain final predicted probability. Extensive qualitative and quantitative experiments on a large-scale publicly available dataset demonstrate the validity and efficiency of our method. Code and models are publicly available at https://github.com/leilimaster/DmifNet.