CD-NGP: A Fast Scalable Continual Representation for Dynamic Scenes
This addresses memory and scalability issues in dynamic scene rendering for applications like VR/AR, though it is incremental as it builds on existing neural graphics primitives.
The paper tackles the problem of novel view synthesis in dynamic scenes by proposing CD-NGP, a continual learning framework that reduces memory usage to <14GB and requires only 0.4MB/frame in streaming bandwidth on DyNeRF, while improving scalability and reconstruction quality.
Novel view synthesis (NVS) in dynamic scenes faces persistent challenges in memory consumption, model complexity, training efficiency, and rendering quality. Offline methods offer high fidelity but suffer from high memory usage and limited scalability, while online approaches often trade quality for speed and compactness. We propose Continual Dynamic Neural Graphics Primitives (CD-NGP), a continual learning framework that reduces memory overhead and enhances scalability through parameter reuse. To avoid feature interference in dynamic scenes and improve rendering quality, our method combines spatial and temporal hash encodings, which compactly represent scene structures and motion patterns. We also introduce a new dataset comprising multi-view, long-duration ($>1200$ frames) videos with both rigid and non-rigid motion, which is not found in existing benchmarks. CD-NGP is evaluated on public datasets and our long video dataset, demonstrating superior scalability and reconstruction quality. It significantly reduces training memory usage to <14GB and requires only 0.4MB/frame in streaming bandwidth on DyNeRF -- substantially lower than most online baselines.