Using Image Priors to Improve Scene Understanding
This work addresses the need for robust scene understanding in autonomous vehicles by incorporating temporal information, representing an incremental improvement over existing methods.
The paper tackles the problem of improving semantic segmentation for autonomous driving by leveraging image priors from sequential data, resulting in accuracy improvements from 69.1% to 73.3% for dynamic classes and from 88.2% to 89.1% for static classes, while achieving the same accuracy with 5 times fewer parameters compared to baseline models.
Semantic segmentation algorithms that can robustly segment objects across multiple camera viewpoints are crucial for assuring navigation and safety in emerging applications such as autonomous driving. Existing algorithms treat each image in isolation, but autonomous vehicles often revisit the same locations or maintain information from the immediate past. We propose a simple yet effective method for leveraging these image priors to improve semantic segmentation of images from sequential driving datasets. We examine several methods to fuse these temporal scene priors, and introduce a prior fusion network that is able to learn how to transfer this information. The prior fusion model improves the accuracy over the non-prior baseline from 69.1% to 73.3% for dynamic classes, and from 88.2% to 89.1% for static classes. Compared to models such as FCN-8, our prior method achieves the same accuracy with 5 times fewer parameters. We used a simple encoder decoder backbone, but this general prior fusion method could be applied to more complex semantic segmentation backbones. We also discuss how structured representations of scenes in the form of a scene graph could be leveraged as priors to further improve scene understanding.