GaussianGAN: Real-Time Photorealistic controllable Human Avatars
This work addresses the need for high-quality, controllable avatars in applications like virtual reality or gaming, representing an incremental improvement over existing methods.
The paper tackled the problem of blurring in photorealistic human avatars by proposing GaussianGAN, which uses Gaussian splatting and a UNet generator to achieve real-time rendering at 79 FPS, with state-of-the-art pixel fidelity scores of 32.94 dB on the ZJU Mocap dataset and 33.39 dB on the Thuman4 dataset.
Photorealistic and controllable human avatars have gained popularity in the research community thanks to rapid advances in neural rendering, providing fast and realistic synthesis tools. However, a limitation of current solutions is the presence of noticeable blurring. To solve this problem, we propose GaussianGAN, an animatable avatar approach developed for photorealistic rendering of people in real-time. We introduce a novel Gaussian splatting densification strategy to build Gaussian points from the surface of cylindrical structures around estimated skeletal limbs. Given the camera calibration, we render an accurate semantic segmentation with our novel view segmentation module. Finally, a UNet generator uses the rendered Gaussian splatting features and the segmentation maps to create photorealistic digital avatars. Our method runs in real-time with a rendering speed of 79 FPS. It outperforms previous methods regarding visual perception and quality, achieving a state-of-the-art results in terms of a pixel fidelity of 32.94db on the ZJU Mocap dataset and 33.39db on the Thuman4 dataset.