What Makes Kevin Spacey Look Like Kevin Spacey
This enables modeling and puppeteering of celebrities without physical scanning, which is useful for entertainment and digital media applications, though it is incremental in combining existing techniques.
The paper tackles the problem of reconstructing a controllable model of a person from unstructured photo collections to capture their persona, enabling puppeteering where one person acts out another's role while retaining their own character. It demonstrates convincing results on celebrities using Internet imagery and video.
We reconstruct a controllable model of a person from a large photo collection that captures his or her {\em persona}, i.e., physical appearance and behavior. The ability to operate on unstructured photo collections enables modeling a huge number of people, including celebrities and other well photographed people without requiring them to be scanned. Moreover, we show the ability to drive or {\em puppeteer} the captured person B using any other video of a different person A. In this scenario, B acts out the role of person A, but retains his/her own personality and character. Our system is based on a novel combination of 3D face reconstruction, tracking, alignment, and multi-texture modeling, applied to the puppeteering problem. We demonstrate convincing results on a large variety of celebrities derived from Internet imagery and video.