Real-time Hair Segmentation and Recoloring on Mobile GPUs
This work solves the problem of efficient hair segmentation for AR effects on mobile devices, with deployment in a major app used by millions, but it is incremental as it builds on existing neural network approaches.
The paper tackles real-time hair segmentation from a single camera input for mobile AR applications, achieving 30-100+ FPS on mobile GPUs with high accuracy and enabling realistic hair recoloring.
We present a novel approach for neural network-based hair segmentation from a single camera input specifically designed for real-time, mobile application. Our relatively small neural network produces a high-quality hair segmentation mask that is well suited for AR effects, e.g. virtual hair recoloring. The proposed model achieves real-time inference speed on mobile GPUs (30-100+ FPS, depending on the device) with high accuracy. We also propose a very realistic hair recoloring scheme. Our method has been deployed in major AR application and is used by millions of users.