DANBO: Disentangled Articulated Neural Body Representations via Graph Neural Networks
This work addresses the challenge of accessible avatar creation for end users by improving generalization and reducing artifacts in pose-dependent deformations, though it is incremental as it builds on existing neural body representation methods.
The paper tackles the problem of creating realistic animatable human avatars from raw images without expensive studio setups, by introducing a three-stage method using graph neural networks and localized per-bone features to reduce spurious correlations and improve pose-dependent deformation, resulting in more plausible body shapes and high-quality image synthesis under unseen poses.
Deep learning greatly improved the realism of animatable human models by learning geometry and appearance from collections of 3D scans, template meshes, and multi-view imagery. High-resolution models enable photo-realistic avatars but at the cost of requiring studio settings not available to end users. Our goal is to create avatars directly from raw images without relying on expensive studio setups and surface tracking. While a few such approaches exist, those have limited generalization capabilities and are prone to learning spurious (chance) correlations between irrelevant body parts, resulting in implausible deformations and missing body parts on unseen poses. We introduce a three-stage method that induces two inductive biases to better disentangled pose-dependent deformation. First, we model correlations of body parts explicitly with a graph neural network. Second, to further reduce the effect of chance correlations, we introduce localized per-bone features that use a factorized volumetric representation and a new aggregation function. We demonstrate that our model produces realistic body shapes under challenging unseen poses and shows high-quality image synthesis. Our proposed representation strikes a better trade-off between model capacity, expressiveness, and robustness than competing methods. Project website: https://lemonatsu.github.io/danbo.