Deep3D: Fully Automatic 2D-to-3D Video Conversion with Deep Convolutional Neural Networks
This addresses the growing demand for 3D content in movies and VR by providing a fully automatic conversion method, though it is incremental as it builds on existing 2D-to-3D conversion techniques.
The paper tackled the problem of automatically converting 2D videos to stereoscopic 3D format using deep neural networks, achieving significant performance improvements over baselines in quantitative and human evaluations.
As 3D movie viewing becomes mainstream and Virtual Reality (VR) market emerges, the demand for 3D contents is growing rapidly. Producing 3D videos, however, remains challenging. In this paper we propose to use deep neural networks for automatically converting 2D videos and images to stereoscopic 3D format. In contrast to previous automatic 2D-to-3D conversion algorithms, which have separate stages and need ground truth depth map as supervision, our approach is trained end-to-end directly on stereo pairs extracted from 3D movies. This novel training scheme makes it possible to exploit orders of magnitude more data and significantly increases performance. Indeed, Deep3D outperforms baselines in both quantitative and human subject evaluations.