READ: Large-Scale Neural Scene Rendering for Autonomous Driving
This addresses the challenge of experimenting with different scenarios in autonomous vehicles, offering a domain-specific solution for generating coherent driving scenes.
The paper tackles the problem of synthesizing free-view photo-realistic images for autonomous driving by proposing a large-scale neural rendering method (READ) that learns neural descriptors from sparse point clouds, enabling realistic scene synthesis, stitching, and editing on a PC.
Synthesizing free-view photo-realistic images is an important task in multimedia. With the development of advanced driver assistance systems~(ADAS) and their applications in autonomous vehicles, experimenting with different scenarios becomes a challenge. Although the photo-realistic street scenes can be synthesized by image-to-image translation methods, which cannot produce coherent scenes due to the lack of 3D information. In this paper, a large-scale neural rendering method is proposed to synthesize the autonomous driving scene~(READ), which makes it possible to synthesize large-scale driving scenarios on a PC through a variety of sampling schemes. In order to represent driving scenarios, we propose an ω rendering network to learn neural descriptors from sparse point clouds. Our model can not only synthesize realistic driving scenes but also stitch and edit driving scenes. Experiments show that our model performs well in large-scale driving scenarios.