Exploring Surround-View Fisheye Camera 3D Object Detection
This addresses the problem of accurate 3D object detection in automotive systems with fisheye cameras, which is incremental as it adapts existing paradigms to a specific domain.
The paper tackled 3D object detection using surround-view fisheye cameras by developing two methods, FisheyeBEVDet and FisheyePETR, that incorporate fisheye geometry, and it released a new dataset, Fisheye3DOD, showing accuracy improvements of up to 6.2% over baselines.
In this work, we explore the technical feasibility of implementing end-to-end 3D object detection (3DOD) with surround-view fisheye camera system. Specifically, we first investigate the performance drop incurred when transferring classic pinhole-based 3D object detectors to fisheye imagery. To mitigate this, we then develop two methods that incorporate the unique geometry of fisheye images into mainstream detection frameworks: one based on the bird's-eye-view (BEV) paradigm, named FisheyeBEVDet, and the other on the query-based paradigm, named FisheyePETR. Both methods adopt spherical spatial representations to effectively capture fisheye geometry. In light of the lack of dedicated evaluation benchmarks, we release Fisheye3DOD, a new open dataset synthesized using CARLA and featuring both standard pinhole and fisheye camera arrays. Experiments on Fisheye3DOD show that our fisheye-compatible modeling improves accuracy by up to 6.2% over baseline methods.