Unsupervised Optical Flow Estimation with Dynamic Timing Representation for Spike Camera
This work addresses optical flow estimation for spike cameras in high-speed scenes like autonomous driving, but it is incremental as it builds on existing spike-based methods.
The paper tackles the problem of efficiently selecting spike stream data length for optical flow estimation by proposing a dynamic timing representation and an unsupervised learning method, achieving 15% and 19% error reduction compared to the best spike-based work in specific settings.
Efficiently selecting an appropriate spike stream data length to extract precise information is the key to the spike vision tasks. To address this issue, we propose a dynamic timing representation for spike streams. Based on multi-layers architecture, it applies dilated convolutions on temporal dimension to extract features on multi-temporal scales with few parameters. And we design layer attention to dynamically fuse these features. Moreover, we propose an unsupervised learning method for optical flow estimation in a spike-based manner to break the dependence on labeled data. In addition, to verify the robustness, we also build a spike-based synthetic validation dataset for extreme scenarios in autonomous driving, denoted as SSES dataset. It consists of various corner cases. Experiments show that our method can predict optical flow from spike streams in different high-speed scenes, including real scenes. For instance, our method gets $15\%$ and $19\%$ error reduction from the best spike-based work, SCFlow, in $Δt=10$ and $Δt=20$ respectively which are the same settings as the previous works.