Hair Segmentation on Time-of-Flight RGBD Images
This work addresses hair segmentation for computer vision applications, offering a novel sensor-based approach that is incremental but provides specific gains in handling difficult cases.
The paper tackles the challenging problem of robust hair segmentation in portrait images by introducing a computational imaging solution that uses Time-of-Flight (ToF) RGBD sensors, achieving improved accuracy and robustness over RGB-based techniques, particularly for cases like dark hair and similar hair/background scenarios.
Robust segmentation of hair from portrait images remains challenging: hair does not conform to a uniform shape, style or even color; dark hair in particular lacks features. We present a novel computational imaging solution that tackles the problem from both input and processing fronts. We explore using Time-of-Flight (ToF) RGBD sensors on recent mobile devices. We first conduct a comprehensive analysis to show that scattering and inter-reflection cause different noise patterns on hair vs. non-hair regions on ToF images, by changing the light path and/or combining multiple paths. We then develop a deep network based approach that employs both ToF depth map and the RGB gradient maps to produce an initial hair segmentation with labeled hair components. We then refine the result by imposing ToF noise prior under the conditional random field. We collect the first ToF RGBD hair dataset with 20k+ head images captured on 30 human subjects with a variety of hairstyles at different view angles. Comprehensive experiments show that our approach outperforms the RGB based techniques in accuracy and robustness and can handle traditionally challenging cases such as dark hair, similar hair/background, similar hair/foreground, etc.