FOUND: Foot Optimization with Uncertain Normals for Surface Deformation Using Synthetic Data
This addresses the problem of 3D foot reconstruction from limited images for applications like healthcare or footwear design, but it is incremental as it builds on existing generative models and optimization techniques.
The paper tackled few-view 3D reconstruction of human feet by developing FOUND, which includes a synthetic dataset, an uncertainty-aware normal predictor, and an optimization scheme, showing that the normal predictor outperforms off-the-shelf methods and the optimization outperforms state-of-the-art photogrammetry pipelines in few-view settings.
Surface reconstruction from multi-view images is a challenging task, with solutions often requiring a large number of sampled images with high overlap. We seek to develop a method for few-view reconstruction, for the case of the human foot. To solve this task, we must extract rich geometric cues from RGB images, before carefully fusing them into a final 3D object. Our FOUND approach tackles this, with 4 main contributions: (i) SynFoot, a synthetic dataset of 50,000 photorealistic foot images, paired with ground truth surface normals and keypoints; (ii) an uncertainty-aware surface normal predictor trained on our synthetic dataset; (iii) an optimization scheme for fitting a generative foot model to a series of images; and (iv) a benchmark dataset of calibrated images and high resolution ground truth geometry. We show that our normal predictor outperforms all off-the-shelf equivalents significantly on real images, and our optimization scheme outperforms state-of-the-art photogrammetry pipelines, especially for a few-view setting. We release our synthetic dataset and baseline 3D scans to the research community.