Image-free single-pixel segmentation
This technique addresses the hardware and software waste in resource-limited platforms like UAVs and unmanned vehicles that require real-time sensing, offering a novel approach to efficient segmentation.
The paper tackles the problem of semantic segmentation requiring high-fidelity images by proposing an image-free single-pixel segmentation technique that combines structured illumination and single-pixel detection to directly infer segmentation maps from compressed measurements. The result is accurate segmentation using two orders of magnitude less input data, achieving a Dice coefficient above 80% and pixel accuracy above 96% at a 1% sampling ratio.
The existing segmentation techniques require high-fidelity images as input to perform semantic segmentation. Since the segmentation results contain most of edge information that is much less than the acquired images, the throughput gap leads to both hardware and software waste. In this letter, we report an image-free single-pixel segmentation technique. The technique combines structured illumination and single-pixel detection together, to efficiently samples and multiplexes scene's segmentation information into compressed one-dimensional measurements. The illumination patterns are optimized together with the subsequent reconstruction neural network, which directly infers segmentation maps from the single-pixel measurements. The end-to-end encoding-and-decoding learning framework enables optimized illumination with corresponding network, which provides both high acquisition and segmentation efficiency. Both simulation and experimental results validate that accurate segmentation can be achieved using two-order-of-magnitude less input data. When the sampling ratio is 1%, the Dice coefficient reaches above 80% and the pixel accuracy reaches above 96%. We envision that this image-free segmentation technique can be widely applied in various resource-limited platforms such as UAV and unmanned vehicle that require real-time sensing.