HumanNeRF-SE: A Simple yet Effective Approach to Animate HumanNeRF with Diverse Poses
This work addresses the challenge of efficient and high-quality human image animation for applications like virtual reality or gaming, though it is incremental as it builds on existing HumanNeRF approaches.
The paper tackles the problem of synthesizing images of humans in diverse poses using few-shot input by combining explicit and implicit human representations to reduce computational complexity. The result is a 15x speed increase in image synthesis and better performance with fewer parameters compared to state-of-the-art methods.
We present HumanNeRF-SE, a simple yet effective method that synthesizes diverse novel pose images with simple input. Previous HumanNeRF works require a large number of optimizable parameters to fit the human images. Instead, we reload these approaches by combining explicit and implicit human representations to design both generalized rigid deformation and specific non-rigid deformation. Our key insight is that explicit shape can reduce the sampling points used to fit implicit representation, and frozen blending weights from SMPL constructing a generalized rigid deformation can effectively avoid overfitting and improve pose generalization performance. Our architecture involving both explicit and implicit representation is simple yet effective. Experiments demonstrate our model can synthesize images under arbitrary poses with few-shot input and increase the speed of synthesizing images by 15 times through a reduction in computational complexity without using any existing acceleration modules. Compared to the state-of-the-art HumanNeRF studies, HumanNeRF-SE achieves better performance with fewer learnable parameters and less training time.