End-to-end optimized image compression for multiple machine tasks
This addresses the need for efficient image compression in computer vision applications where compressed data must serve multiple tasks, offering a practical solution for remote analysis.
The paper tackles the problem of compressing images for multiple machine tasks by introducing 'Connectors' that transform compressed content optimized for one task to others, achieving significant rate-accuracy improvements for image classification and object segmentation using a bit-stream originally optimized for object detection.
An increasing share of captured images and videos are transmitted for storage and remote analysis by computer vision algorithms, rather than to be viewed by humans. Contrary to traditional standard codecs with engineered tools, neural network based codecs can be trained end-to-end to optimally compress images with respect to a target rate and any given differentiable performance metric. Although it is possible to train such compression tools to achieve better rate-accuracy performance for a particular computer vision task, it could be practical and relevant to re-use the compressed bit-stream for multiple machine tasks. For this purpose, we introduce 'Connectors' that are inserted between the decoder and the task algorithms to enable a direct transformation of the compressed content, which was previously optimized for a specific task, to multiple other machine tasks. We demonstrate the effectiveness of the proposed method by achieving significant rate-accuracy performance improvement for both image classification and object segmentation, using the same bit-stream, originally optimized for object detection.