Towards improved lossy image compression: Human image reconstruction with public-domain images
This work addresses the challenge of improving image compression for human perception by leveraging human reconstruction and public datasets, though it appears incremental as it builds on existing compression methods with a new evaluation approach.
The paper tackled the problem of unsatisfying results in lossy image compression at low bitrates by introducing a human-centric paradigm where one human describes images to another who reconstructs them using public-domain images and text instructions, with results evaluated by human raters on Amazon Mechanical Turk and compared to WebP, suggesting that prioritizing semantic visual elements can lead to improvements.
Lossy image compression has been studied extensively in the context of typical loss functions such as RMSE, MS-SSIM, etc. However, compression at low bitrates generally produces unsatisfying results. Furthermore, the availability of massive public image datasets appears to have hardly been exploited in image compression. Here, we present a paradigm for eliciting human image reconstruction in order to perform lossy image compression. In this paradigm, one human describes images to a second human, whose task is to reconstruct the target image using publicly available images and text instructions. The resulting reconstructions are then evaluated by human raters on the Amazon Mechanical Turk platform and compared to reconstructions obtained using state-of-the-art compressor WebP. Our results suggest that prioritizing semantic visual elements may be key to achieving significant improvements in image compression, and that our paradigm can be used to develop a more human-centric loss function. The images, results and additional data are available at https://compression.stanford.edu/human-compression