Incorporating Season and Solar Specificity into Renderings made by a NeRF Architecture using Satellite Images
This work addresses the challenge of creating more accurate and realistic satellite image renderings for remote sensing applications, but it is incremental as it builds on prior NeRF extensions like Shadow NeRF and Sat-NeRF.
The paper tackles the problem of making Neural Radiance Field (NeRF) renderings from satellite images season-specific while handling shadows, by introducing time of year as an input and adding loss terms to separate seasonal features from shadows. It demonstrates performance on eight Areas of Interest using Maxar WorldView-3 satellite data, with evaluations on novel view rendering, height maps, shadow prediction, and seasonal feature specification.
As a result of Shadow NeRF and Sat-NeRF, it is possible to take the solar angle into account in a NeRF-based framework for rendering a scene from a novel viewpoint using satellite images for training. Our work extends those contributions and shows how one can make the renderings season-specific. Our main challenge was creating a Neural Radiance Field (NeRF) that could render seasonal features independently of viewing angle and solar angle while still being able to render shadows. We teach our network to render seasonal features by introducing one more input variable -- time of the year. However, the small training datasets typical of satellite imagery can introduce ambiguities in cases where shadows are present in the same location for every image of a particular season. We add additional terms to the loss function to discourage the network from using seasonal features for accounting for shadows. We show the performance of our network on eight Areas of Interest containing images captured by the Maxar WorldView-3 satellite. This evaluation includes tests measuring the ability of our framework to accurately render novel views, generate height maps, predict shadows, and specify seasonal features independently from shadows. Our ablation studies justify the choices made for network design parameters.