GPU-Accelerated Mobile Multi-view Style Transfer
This work addresses the need for efficient and consistent artistic style transfer tools in 3D photo platforms on smartphones, enabling user-generated content creation.
The paper tackles the problem of applying neural style transfer to multi-view images on mobile devices, which previously produced inconsistent results and was too slow, by presenting a GPU-accelerated pipeline that enforces style consistency and achieves on-demand performance.
An estimated 60% of smartphones sold in 2018 were equipped with multiple rear cameras, enabling a wide variety of 3D-enabled applications such as 3D Photos. The success of 3D Photo platforms (Facebook 3D Photo, Holopix, etc) depend on a steady influx of user generated content. These platforms must provide simple image manipulation tools to facilitate content creation, akin to traditional photo platforms. Artistic neural style transfer, propelled by recent advancements in GPU technology, is one such tool for enhancing traditional photos. However, naively extrapolating single-view neural style transfer to the multi-view scenario produces visually inconsistent results and is prohibitively slow on mobile devices. We present a GPU-accelerated multi-view style transfer pipeline which enforces style consistency between views with on-demand performance on mobile platforms. Our pipeline is modular and creates high quality depth and parallax effects from a stereoscopic image pair.