A photosensor employing data-driven binning for ultrafast image recognition
This work addresses the need for ultrafast image recognition in optical sensing applications, representing an incremental improvement over traditional binning techniques.
The researchers tackled the problem of reducing data processing and noise in image sensors by developing a data-driven binning method that combines most sensor elements into a single superpixel optimized via machine learning, achieving nanosecond-scale classification of MNIST images without accuracy loss.
Pixel binning is a technique, widely used in optical image acquisition and spectroscopy, in which adjacent detector elements of an image sensor are combined into larger pixels. This reduces the amount of data to be processed as well as the impact of noise, but comes at the cost of a loss of information. Here, we push the concept of binning to its limit by combining a large fraction of the sensor elements into a single superpixel that extends over the whole face of the chip. For a given pattern recognition task, its optimal shape is determined from training data using a machine learning algorithm. We demonstrate the classification of optically projected images from the MNIST dataset on a nanosecond timescale, with enhanced sensitivity and without loss of classification accuracy. Our concept is not limited to imaging alone but can also be applied in optical spectroscopy or other sensing applications.