What's in the Image? Explorable Decoding of Compressed Images
This addresses the ambiguity in decompressed images for users in fields like forensics and medicine, but it is incremental as it builds on existing JPEG compression.
The paper tackles the problem of lossy image compression discarding semantically meaningful information, especially at low bit rates, by proposing the first method that allows users to explore multiple possible decompressed images consistent with a compressed JPEG file, using a novel deep-network decoder and graphical interface. It demonstrates applications in graphical, medical, and forensic use cases.
The ever-growing amounts of visual contents captured on a daily basis necessitate the use of lossy compression methods in order to save storage space and transmission bandwidth. While extensive research efforts are devoted to improving compression techniques, every method inevitably discards information. Especially at low bit rates, this information often corresponds to semantically meaningful visual cues, so that decompression involves significant ambiguity. In spite of this fact, existing decompression algorithms typically produce only a single output, and do not allow the viewer to explore the set of images that map to the given compressed code. In this work we propose the first image decompression method to facilitate user-exploration of the diverse set of natural images that could have given rise to the compressed input code, thus granting users the ability to determine what could and what could not have been there in the original scene. Specifically, we develop a novel deep-network based decoder architecture for the ubiquitous JPEG standard, which allows traversing the set of decompressed images that are consistent with the compressed JPEG file. To allow for simple user interaction, we develop a graphical user interface comprising several intuitive exploration tools, including an automatic tool for examining specific solutions of interest. We exemplify our framework on graphical, medical and forensic use cases, demonstrating its wide range of potential applications.