PrismAvatar: Real-time animated 3D neural head avatars on edge devices
This work addresses the problem of real-time 3D avatar rendering for users of edge devices, such as mobile devices, providing an incremental solution for real-time animation and rendering.
PrismAvatar tackles the problem of real-time animated 3D neural head avatars on edge devices, achieving 60 fps with low memory usage on mobile devices and comparable quality to state-of-the-art 3D avatar models. The result enables efficient rendering and animation on resource-constrained devices.
We present PrismAvatar: a 3D head avatar model which is designed specifically to enable real-time animation and rendering on resource-constrained edge devices, while still enjoying the benefits of neural volumetric rendering at training time. By integrating a rigged prism lattice with a 3D morphable head model, we use a hybrid rendering model to simultaneously reconstruct a mesh-based head and a deformable NeRF model for regions not represented by the 3DMM. We then distill the deformable NeRF into a rigged mesh and neural textures, which can be animated and rendered efficiently within the constraints of the traditional triangle rendering pipeline. In addition to running at 60 fps with low memory usage on mobile devices, we find that our trained models have comparable quality to state-of-the-art 3D avatar models on desktop devices.