Secrets of Edge-Informed Contrast Maximization for Event-Based Vision
This work addresses the challenge of improving motion estimation accuracy in event-based vision, which is crucial for applications like robotics and autonomous systems, though it is incremental as it builds on existing contrast maximization methods.
The paper tackles the problem of reconstructing sharp spatial structures from event camera data by extending contrast maximization to incorporate edge information, achieving new state-of-the-art results on optical flow benchmarks with superior sharpness scores.
Event cameras capture the motion of intensity gradients (edges) in the image plane in the form of rapid asynchronous events. When accumulated in 2D histograms, these events depict overlays of the edges in motion, consequently obscuring the spatial structure of the generating edges. Contrast maximization (CM) is an optimization framework that can reverse this effect and produce sharp spatial structures that resemble the moving intensity gradients by estimating the motion trajectories of the events. Nonetheless, CM is still an underexplored area of research with avenues for improvement. In this paper, we propose a novel hybrid approach that extends CM from uni-modal (events only) to bi-modal (events and edges). We leverage the underpinning concept that, given a reference time, optimally warped events produce sharp gradients consistent with the moving edge at that time. Specifically, we formalize a correlation-based objective to aid CM and provide key insights into the incorporation of multiscale and multireference techniques. Moreover, our edge-informed CM method yields superior sharpness scores and establishes new state-of-the-art event optical flow benchmarks on the MVSEC, DSEC, and ECD datasets.