NBAvatar: Neural Billboards Avatars with Realistic Hand-Face Interaction
This addresses the challenge of creating more lifelike digital avatars for applications like virtual reality or gaming, though it appears incremental as it builds on prior avatar rendering techniques.
The paper tackles the problem of realistic rendering of head avatars with non-rigid deformations from hand-face interactions, achieving up to 30% LPIPS reduction and improvements in PSNR and SSIM compared to existing methods.
We present NBAvatar - a method for realistic rendering of head avatars handling non-rigid deformations caused by hand-face interaction. We introduce a novel representation for animated avatars by combining the training of oriented planar primitives with neural rendering. Such a combination of explicit and implicit representations enables NBAvatar to handle temporally and pose-consistent geometry, along with fine-grained appearance details provided by the neural rendering technique. In our experiments, we demonstrate that NBAvatar implicitly learns color transformations caused by face-hand interactions and surpasses existing approaches in terms of novel-view and novel-pose rendering quality. Specifically, NBAvatar achieves up to 30% LPIPS reduction under high-resolution megapixel rendering compared to Gaussian-based avatar methods, while also improving PSNR and SSIM, and achieves higher structural similarity compared to the state-of-the-art hand-face interaction method InteractAvatar.