Synthesizing Images of Humans in Unseen Poses
This addresses the computational challenge of human pose synthesis for applications like animation or virtual reality, but it is incremental as it builds on existing generative methods with a modular approach.
The paper tackles the problem of generating images of a person in new poses from a single input image, using a modular neural network that separates and moves body parts, then composites them with a background. It demonstrates accurate results on golf, yoga, and tennis action classes, producing coherent videos from pose sequences.
We address the computational problem of novel human pose synthesis. Given an image of a person and a desired pose, we produce a depiction of that person in that pose, retaining the appearance of both the person and background. We present a modular generative neural network that synthesizes unseen poses using training pairs of images and poses taken from human action videos. Our network separates a scene into different body part and background layers, moves body parts to new locations and refines their appearances, and composites the new foreground with a hole-filled background. These subtasks, implemented with separate modules, are trained jointly using only a single target image as a supervised label. We use an adversarial discriminator to force our network to synthesize realistic details conditioned on pose. We demonstrate image synthesis results on three action classes: golf, yoga/workouts and tennis, and show that our method produces accurate results within action classes as well as across action classes. Given a sequence of desired poses, we also produce coherent videos of actions.