SQ-GAN: Semantic Image Communications Using Masked Vector Quantization
This work addresses the problem of efficient image compression for task-oriented communications, which is significant for applications where preserving semantic information is crucial.
The authors tackled the problem of semantic image communications by introducing SQ-GAN, which achieves state-of-the-art image compression at extremely low compression rates, outperforming methods like JPEG2000 and BPG. SQ-GAN improves metrics such as perceptual quality and semantic segmentation accuracy on the reconstructed image.
This work introduces Semantically Masked Vector Quantized Generative Adversarial Network (SQ-GAN), a novel approach integrating semantically driven image coding and vector quantization to optimize image compression for semantic/task-oriented communications. The method only acts on source coding and is fully compliant with legacy systems. The semantics is extracted from the image computing its semantic segmentation map using off-the-shelf software. A new specifically developed semantic-conditioned adaptive mask module (SAMM) selectively encodes semantically relevant features of the image. The relevance of the different semantic classes is task-specific, and it is incorporated in the training phase by introducing appropriate weights in the loss function. SQ-GAN outperforms state-of-the-art image compression schemes such as JPEG2000, BPG, and deep-learning based methods across multiple metrics, including perceptual quality and semantic segmentation accuracy on the reconstructed image, at extremely low compression rates.