Out-of-Distribution Robustness in Deep Learning Compression
This addresses robustness issues in compression systems for real-world applications, but it is incremental as it builds on existing compression methods.
The paper tackles the problem of deep neural network compressors being vulnerable to out-of-distribution data, proposing two methods (distributionally-robust optimization and structured latent codes) that enforce robustness compared to standard compressors, with the structured code showing superiority in some cases.
In recent years, deep neural network (DNN) compression systems have proved to be highly effective for designing source codes for many natural sources. However, like many other machine learning systems, these compressors suffer from vulnerabilities to distribution shifts as well as out-of-distribution (OOD) data, which reduces their real-world applications. In this paper, we initiate the study of OOD robust compression. Considering robustness to two types of ambiguity sets (Wasserstein balls and group shifts), we propose algorithmic and architectural frameworks built on two principled methods: one that trains DNN compressors using distributionally-robust optimization (DRO), and the other which uses a structured latent code. Our results demonstrate that both methods enforce robustness compared to a standard DNN compressor, and that using a structured code can be superior to the DRO compressor. We observe tradeoffs between robustness and distortion and corroborate these findings theoretically for a specific class of sources.