Deep Manifold Traversal: Changing Labels with Convolutional Features
It addresses a general-purpose method for semantic image editing, which is incremental as it builds on existing manifold and deep learning techniques.
The paper tackles the general problem of 'label changing' in computer vision by proposing deep manifold traversal, which approximates the natural image manifold and morphs images along a path between source and target classes, demonstrating effectiveness on diverse tasks like altering age, hair color, season, and city skyline transformations.
Many tasks in computer vision can be cast as a "label changing" problem, where the goal is to make a semantic change to the appearance of an image or some subject in an image in order to alter the class membership. Although successful task-specific methods have been developed for some label changing applications, to date no general purpose method exists. Motivated by this we propose deep manifold traversal, a method that addresses the problem in its most general form: it first approximates the manifold of natural images then morphs a test image along a traversal path away from a source class and towards a target class while staying near the manifold throughout. The resulting algorithm is surprisingly effective and versatile. It is completely data driven, requiring only an example set of images from the desired source and target domains. We demonstrate deep manifold traversal on highly diverse label changing tasks: changing an individual's appearance (age and hair color), changing the season of an outdoor image, and transforming a city skyline towards nighttime.