Low-Rank Adaptation of Neural Fields
This work addresses a specific challenge in graphics and neural representation for tasks like video compression and editing, but it is incremental as it adapts an existing method from LLMs to neural fields.
The paper tackles the problem of efficiently encoding small changes to neural fields, such as for image filtering or geometry editing, by proposing a low-rank adaptation strategy that yields lightweight updates with minimal computational overhead.
Processing visual data often involves small adjustments or sequences of changes, e.g., image filtering, surface smoothing, and animation. While established graphics techniques like normal mapping and video compression exploit redundancy to encode such small changes efficiently, the problem of encoding small changes to neural fields -- neural network parameterizations of visual or physical functions -- has received less attention. We propose a parameter-efficient strategy for updating neural fields using low-rank adaptations (LoRA). LoRA, a method from the parameter-efficient fine-tuning LLM community, encodes small updates to pre-trained models with minimal computational overhead. We adapt LoRA for instance-specific neural fields, avoiding the need for large pre-trained models and yielding lightweight updates. We validate our approach with experiments in image filtering, geometry editing, video compression, and energy-based editing, demonstrating its effectiveness and versatility for representing neural field updates.