Representation Learning for Event-based Visuomotor Policies
This work addresses the problem of high-speed navigation for robotics by enabling more efficient and robust visuomotor policies using event-based cameras, representing an incremental improvement over existing techniques.
The paper tackles the challenge of using asynchronous event-based camera data for machine learning by introducing an event variational autoencoder to learn compact representations, which are then applied to reinforcement learning for obstacle avoidance, resulting in faster policy training, adaptability to control capacities, and improved robustness compared to image-based methods.
Event-based cameras are dynamic vision sensors that provide asynchronous measurements of changes in per-pixel brightness at a microsecond level. This makes them significantly faster than conventional frame-based cameras, and an appealing choice for high-speed navigation. While an interesting sensor modality, this asynchronously streamed event data poses a challenge for machine learning techniques that are more suited for frame-based data. In this paper, we present an event variational autoencoder and show that it is feasible to learn compact representations directly from asynchronous spatiotemporal event data. Furthermore, we show that such pretrained representations can be used for event-based reinforcement learning instead of end-to-end reward driven perception. We validate this framework of learning event-based visuomotor policies by applying it to an obstacle avoidance scenario in simulation. Compared to techniques that treat event data as images, we show that representations learnt from event streams result in faster policy training, adapt to different control capacities, and demonstrate a higher degree of robustness.