Local Geometric Indexing of High Resolution Data for Facial Reconstruction from Sparse Markers
This addresses the challenge of accurate facial animation for motion capture applications, though it appears incremental as it builds on existing data-driven methods.
The paper tackles the problem of facial reconstruction from sparse motion capture markers by avoiding overfitting or underfitting through a local geometric indexing approach that matches markers to instances in a high-resolution dataset, augmented with targeted physical simulations.
When considering sparse motion capture marker data, one typically struggles to balance its overfitting via a high dimensional blendshape system versus underfitting caused by smoothness constraints. With the current trend towards using more and more data, our aim is not to fit the motion capture markers with a parameterized (blendshape) model or to smoothly interpolate a surface through the marker positions, but rather to find an instance in the high resolution dataset that contains local geometry to fit each marker. Just as is true for typical machine learning applications, this approach benefits from a plethora of data, and thus we also consider augmenting the dataset via specially designed physical simulations that target the high resolution dataset such that the simulation output lies on the same so-called manifold as the data targeted.