Content-Consistent Generation of Realistic Eyes with Style
It addresses the scarcity of accurately labeled real-world training data for eye-related tasks, offering a method to enhance datasets with person-specific synthetic images.
The paper tackles the problem of generating realistic eye images that match a given semantic segmentation mask while adopting the style of a specific person from few references, introducing two approaches including one that won the OpenEDS Synthetic Eye Generation Challenge at ICCV 2019.
Accurately labeled real-world training data can be scarce, and hence recent works adapt, modify or generate images to boost target datasets. However, retaining relevant details from input data in the generated images is challenging and failure could be critical to the performance on the final task. In this work, we synthesize person-specific eye images that satisfy a given semantic segmentation mask (content), while following the style of a specified person from only a few reference images. We introduce two approaches, (a) one used to win the OpenEDS Synthetic Eye Generation Challenge at ICCV 2019, and (b) a principled approach to solving the problem involving simultaneous injection of style and content information at multiple scales. Our implementation is available at https://github.com/mcbuehler/Seg2Eye.