EventPoint: Self-Supervised Interest Point Detection and Description for Event-based Camera
This work addresses the challenge of applying local features to event streams for tracking and registration, which is important for robotics and low-power vision systems, representing a novel approach in event-based vision.
The paper tackles the problem of local feature detection and description for event-based cameras by proposing EventPoint, a self-supervised method that uses a new event stream representation called Tencode and neural networks, achieving state-of-the-art performance in feature point detection and matching on public datasets like DSEC, N-Caltech101, and HVGA ATIS Corner Dataset.
This paper proposes a self-supervised learned local detector and descriptor, called EventPoint, for event stream/camera tracking and registration. Event-based cameras have grown in popularity because of their biological inspiration and low power consumption. Despite this, applying local features directly to the event stream is difficult due to its peculiar data structure. We propose a new time-surface-like event stream representation method called Tencode. The event stream data processed by Tencode can obtain the pixel-level positioning of interest points while also simultaneously extracting descriptors through a neural network. Instead of using costly and unreliable manual annotation, our network leverages the prior knowledge of local feature extraction on color images and conducts self-supervised learning via homographic and spatio-temporal adaptation. To the best of our knowledge, our proposed method is the first research on event-based local features learning using a deep neural network. We provide comprehensive experiments of feature point detection and matching, and three public datasets are used for evaluation (i.e. DSEC, N-Caltech101, and HVGA ATIS Corner Dataset). The experimental findings demonstrate that our method outperforms SOTA in terms of feature point detection and description.