Learning Dense Correspondence from Synthetic Environments
This addresses the challenge of mapping human shape from images to 3D models for computer vision applications, but it is incremental as it builds on existing synthetic data methods.
The paper tackles the problem of data scarcity in 2D-3D human mapping by training algorithms on automatically generated synthetic data with known dense correspondences, showing that this approach is a viable alternative to using real data as evaluated on the COCO dataset.
Estimation of human shape and pose from a single image is a challenging task. It is an even more difficult problem to map the identified human shape onto a 3D human model. Existing methods map manually labelled human pixels in real 2D images onto the 3D surface, which is prone to human error, and the sparsity of available annotated data often leads to sub-optimal results. We propose to solve the problem of data scarcity by training 2D-3D human mapping algorithms using automatically generated synthetic data for which exact and dense 2D-3D correspondence is known. Such a learning strategy using synthetic environments has a high generalisation potential towards real-world data. Using different camera parameter variations, background and lighting settings, we created precise ground truth data that constitutes a wider distribution. We evaluate the performance of models trained on synthetic using the COCO dataset and validation framework. Results show that training 2D-3D mapping network models on synthetic data is a viable alternative to using real data.