Annotated Hands for Generative Models
This addresses a specific limitation in image generation for applications requiring realistic human hands, though it appears incremental in scope.
The paper tackles the problem of generative models producing poor-quality hand images by proposing a training framework that augments images with hand annotation channels, resulting in substantially improved hand generation quality as measured by higher confidence in finger joint identification.
Generative models such as GANs and diffusion models have demonstrated impressive image generation capabilities. Despite these successes, these systems are surprisingly poor at creating images with hands. We propose a novel training framework for generative models that substantially improves the ability of such systems to create hand images. Our approach is to augment the training images with three additional channels that provide annotations to hands in the image. These annotations provide additional structure that coax the generative model to produce higher quality hand images. We demonstrate this approach on two different generative models: a generative adversarial network and a diffusion model. We demonstrate our method both on a new synthetic dataset of hand images and also on real photographs that contain hands. We measure the improved quality of the generated hands through higher confidence in finger joint identification using an off-the-shelf hand detector.