DRSM: efficient neural 4d decomposition for dynamic reconstruction in stationary monocular cameras
This addresses the challenge of dynamic scene reconstruction for applications like video sharing and live broadcasting, but it is incremental as it builds on neural rendering techniques.
The paper tackles the problem of reconstructing dynamic scenes from stationary monocular videos, which is under-constrained due to single-view limitations, and presents a novel framework using decomposed static and dynamic feature planes with dense ray casting, achieving higher-fidelity results than existing methods.
With the popularity of monocular videos generated by video sharing and live broadcasting applications, reconstructing and editing dynamic scenes in stationary monocular cameras has become a special but anticipated technology. In contrast to scene reconstructions that exploit multi-view observations, the problem of modeling a dynamic scene from a single view is significantly more under-constrained and ill-posed. Inspired by recent progress in neural rendering, we present a novel framework to tackle 4D decomposition problem for dynamic scenes in monocular cameras. Our framework utilizes decomposed static and dynamic feature planes to represent 4D scenes and emphasizes the learning of dynamic regions through dense ray casting. Inadequate 3D clues from a single-view and occlusion are also particular challenges in scene reconstruction. To overcome these difficulties, we propose deep supervised optimization and ray casting strategies. With experiments on various videos, our method generates higher-fidelity results than existing methods for single-view dynamic scene representation.