FB-BEV: BEV Representation from Forward-Backward View Transformations
This addresses a crucial bottleneck in autonomous driving perception systems by improving BEV representation quality, though it appears incremental as it combines existing paradigms.
The paper tackles the problem of generating high-quality Bird-Eye-View (BEV) representations in camera-based perception systems by proposing a forward-backward view transformation module that compensates for limitations in existing forward and backward projection methods, achieving a state-of-the-art result of 62.4% NDS on the nuScenes test set.
View Transformation Module (VTM), where transformations happen between multi-view image features and Bird-Eye-View (BEV) representation, is a crucial step in camera-based BEV perception systems. Currently, the two most prominent VTM paradigms are forward projection and backward projection. Forward projection, represented by Lift-Splat-Shoot, leads to sparsely projected BEV features without post-processing. Backward projection, with BEVFormer being an example, tends to generate false-positive BEV features from incorrect projections due to the lack of utilization on depth. To address the above limitations, we propose a novel forward-backward view transformation module. Our approach compensates for the deficiencies in both existing methods, allowing them to enhance each other to obtain higher quality BEV representations mutually. We instantiate the proposed module with FB-BEV, which achieves a new state-of-the-art result of 62.4% NDS on the nuScenes test set. Code and models are available at https://github.com/NVlabs/FB-BEV.