Direct Servo Control from In-Sensor CNN Inference with A Pixel Processor Array
This addresses the perception-action gap in robotics by enabling high-speed, efficient control from in-sensor inference, though it is incremental as it applies existing methods to a specific hardware setup.
The paper tackled direct visual sensory-motor control by using a SCAMP-5 Pixel Processor Array (PPA) to run a binary CNN for rock, paper, scissors classification at over 8000 FPS, enabling direct servo control without intermediate hardware.
This work demonstrates direct visual sensory-motor control using high-speed CNN inference via a SCAMP-5 Pixel Processor Array (PPA). We demonstrate how PPAs are able to efficiently bridge the gap between perception and action. A binary Convolutional Neural Network (CNN) is used for a classic rock, paper, scissors classification problem at over 8000 FPS. Control instructions are directly sent to a servo motor from the PPA according to the CNN's classification result without any other intermediate hardware.