Event-based Non-Rigid Reconstruction from Contours
This addresses the challenge of visual reconstruction for fast deformations, which is a problem for robotics or motion analysis, but it is incremental as it builds on existing event-based and reconstruction techniques.
The paper tackles reconstructing fast non-rigid object deformations by using event-based cameras, achieving advantages over state-of-the-art methods in experiments on synthetic and real data for human hand motion.
Visual reconstruction of fast non-rigid object deformations over time is a challenge for conventional frame-based cameras. In this paper, we propose a novel approach for reconstructing such deformations using measurements from event-based cameras. Under the assumption of a static background, where all events are generated by the motion, our approach estimates the deformation of objects from events generated at the object contour in a probabilistic optimization framework. It associates events to mesh faces on the contour and maximizes the alignment of the line of sight through the event pixel with the associated face. In experiments on synthetic and real data, we demonstrate the advantages of our method over state-of-the-art optimization and learning-based approaches for reconstructing the motion of human hands. A video of the experiments is available at https://youtu.be/gzfw7i5OKjg