INT: Towards Infinite-frames 3D Detection with An Efficient Framework
This addresses the problem of inefficient multi-frame processing for real-time 3D detection in autonomous driving, offering a significant improvement over incremental methods.
The paper tackles the computational and memory cost of multi-frame 3D detection by proposing INT, an efficient framework that theoretically uses infinite frames with single-frame computation, achieving a 7% performance boost on Waymo and 15% on nuScenes with minimal latency overhead.
It is natural to construct a multi-frame instead of a single-frame 3D detector for a continuous-time stream. Although increasing the number of frames might improve performance, previous multi-frame studies only used very limited frames to build their systems due to the dramatically increased computational and memory cost. To address these issues, we propose a novel on-stream training and prediction framework that, in theory, can employ an infinite number of frames while keeping the same amount of computation as a single-frame detector. This infinite framework (INT), which can be used with most existing detectors, is utilized, for example, on the popular CenterPoint, with significant latency reductions and performance improvements. We've also conducted extensive experiments on two large-scale datasets, nuScenes and Waymo Open Dataset, to demonstrate the scheme's effectiveness and efficiency. By employing INT on CenterPoint, we can get around 7% (Waymo) and 15% (nuScenes) performance boost with only 2~4ms latency overhead, and currently SOTA on the Waymo 3D Detection leaderboard.