Meta-Continual Learning of Neural Fields
This work addresses the challenge of efficiently adapting neural fields to new tasks without forgetting previous ones, which is incremental as it builds on existing continual learning and neural field methods.
The paper tackles the problem of catastrophic forgetting and slow convergence in continual learning of neural fields by introducing a meta-continual learning strategy with a modular architecture and optimization-based meta-learning, achieving high-quality reconstruction and significantly improved learning speed across six diverse datasets, including rapid adaptation for city-scale NeRF rendering with reduced parameters.
Neural Fields (NF) have gained prominence as a versatile framework for complex data representation. This work unveils a new problem setting termed \emph{Meta-Continual Learning of Neural Fields} (MCL-NF) and introduces a novel strategy that employs a modular architecture combined with optimization-based meta-learning. Focused on overcoming the limitations of existing methods for continual learning of neural fields, such as catastrophic forgetting and slow convergence, our strategy achieves high-quality reconstruction with significantly improved learning speed. We further introduce Fisher Information Maximization loss for neural radiance fields (FIM-NeRF), which maximizes information gains at the sample level to enhance learning generalization, with proved convergence guarantee and generalization bound. We perform extensive evaluations across image, audio, video reconstruction, and view synthesis tasks on six diverse datasets, demonstrating our method's superiority in reconstruction quality and speed over existing MCL and CL-NF approaches. Notably, our approach attains rapid adaptation of neural fields for city-scale NeRF rendering with reduced parameter requirement.