Predictive Temporal Attention on Event-based Video Stream for Energy-efficient Situation Awareness
This work addresses energy efficiency for embedded systems using event-based vision, though it is incremental as it builds on existing neuromorphic and predictive coding models.
The paper tackled the power consumption bottleneck in off-chip communication between DVS cameras and processors by proposing a predictive temporal attention mechanism, which reduced data communication by 46.7% and computation activities by 43.8%.
The Dynamic Vision Sensor (DVS) is an innovative technology that efficiently captures and encodes visual information in an event-driven manner. By combining it with event-driven neuromorphic processing, the sparsity in DVS camera output can result in high energy efficiency. However, similar to many embedded systems, the off-chip communication between the camera and processor presents a bottleneck in terms of power consumption. Inspired by the predictive coding model and expectation suppression phenomenon found in human brain, we propose a temporal attention mechanism to throttle the camera output and pay attention to it only when the visual events cannot be well predicted. The predictive attention not only reduces power consumption in the sensor-processor interface but also effectively decreases the computational workload by filtering out noisy events. We demonstrate that the predictive attention can reduce 46.7% of data communication between the camera and the processor and reduce 43.8% computation activities in the processor.