S2CFormer: Revisiting the RD-Latency Trade-off in Transformer-based Learned Image Compression
This work addresses a critical efficiency bottleneck in learned image compression for applications requiring high-speed decoding, representing a significant but incremental improvement over existing methods.
The paper tackles the suboptimal trade-off between decoding latency and rate-distortion performance in transformer-based learned image compression by proposing the S2CFormer paradigm, which simplifies spatial operations and enhances channel aggregation, resulting in state-of-the-art performance and faster decoding speeds on datasets like Kodak, Tecnick, and CLIC Professional Validation.
Transformer-based Learned Image Compression (LIC) suffers from a suboptimal trade-off between decoding latency and rate-distortion (R-D) performance. Moreover, the critical role of the FeedForward Network (FFN)-based channel aggregation module has been largely overlooked. Our research reveals that efficient channel aggregation-rather than complex and time-consuming spatial operations-is the key to achieving competitive LIC models. Based on this insight, we initiate the ``S2CFormer'' paradigm, a general architecture that simplifies spatial operations and enhances channel operations to overcome the previous trade-off. We present two instances of the S2CFormer: S2C-Conv, and S2C-Attention. Both models demonstrate state-of-the-art (SOTA) R-D performance and significantly faster decoding speed. Furthermore, we introduce S2C-Hybrid, an enhanced variant that maximizes the strengths of different S2CFormer instances to achieve a better performance-latency trade-off. This model outperforms all the existing methods on the Kodak, Tecnick, and CLIC Professional Validation datasets, setting a new benchmark for efficient and high-performance LIC. The code is at \href{https://github.com/YunuoChen/S2CFormer}{https://github.com/YunuoChen/S2CFormer}.