LongShortNet: Exploring Temporal and Semantic Features Fusion in Streaming Perception
This work addresses the latency-accuracy trade-off in autonomous driving perception, offering an incremental improvement by extending long-term temporal modeling to streaming perception.
The paper tackles the problem of streaming perception in autonomous driving by proposing LongShortNet, a dual-path network that fuses long-term temporal motion with short-term spatial semantics, achieving state-of-the-art performance on the Argoverse-HD dataset with minimal computational overhead.
Streaming perception is a critical task in autonomous driving that requires balancing the latency and accuracy of the autopilot system. However, current methods for streaming perception are limited as they only rely on the current and adjacent two frames to learn movement patterns. This restricts their ability to model complex scenes, often resulting in poor detection results. To address this limitation, we propose LongShortNet, a novel dual-path network that captures long-term temporal motion and integrates it with short-term spatial semantics for real-time perception. LongShortNet is notable as it is the first work to extend long-term temporal modeling to streaming perception, enabling spatiotemporal feature fusion. We evaluate LongShortNet on the challenging Argoverse-HD dataset and demonstrate that it outperforms existing state-of-the-art methods with almost no additional computational cost.