Federated Learning for Large-Scale Scene Modeling with Neural Radiance Fields
This work addresses incremental improvements in federated learning for domain-specific applications like large-scale scene modeling, benefiting fields such as autonomous vehicles and drones.
The paper tackles the scalability and maintainability issues of large-scale neural radiance fields (NeRF) for earth-scale scene modeling by proposing a federated learning pipeline with global pose alignment, demonstrating effectiveness on the Mill19 dataset.
We envision a system to continuously build and maintain a map based on earth-scale neural radiance fields (NeRF) using data collected from vehicles and drones in a lifelong learning manner. However, existing large-scale modeling by NeRF has problems in terms of scalability and maintainability when modeling earth-scale environments. Therefore, to address these problems, we propose a federated learning pipeline for large-scale modeling with NeRF. We tailor the model aggregation pipeline in federated learning for NeRF, thereby allowing local updates of NeRF. In the aggregation step, the accuracy of the clients' global pose is critical. Thus, we also propose global pose alignment to align the noisy global pose of clients before the aggregation step. In experiments, we show the effectiveness of the proposed pose alignment and the federated learning pipeline on the large-scale scene dataset, Mill19.