Leveraging Second-Order Curvature for Efficient Learned Image Compression: Theory and Empirical Evidence
This work addresses efficiency and deployability issues in learned image compression, offering a practical drop-in solution for researchers and practitioners, though it is incremental as it applies an existing optimization method to a specific domain.
The paper tackled the problem of slow convergence and suboptimal performance in learned image compression due to gradient conflicts in first-order optimizers, and demonstrated that using a second-order quasi-Newton optimizer (SOAP) improves training efficiency and final performance, with models showing fewer outliers and better robustness to quantization.
Training learned image compression (LIC) models entails navigating a challenging optimization landscape defined by the fundamental trade-off between rate and distortion. Standard first-order optimizers, such as SGD and Adam, struggle with \emph{gradient conflicts} arising from competing objectives, leading to slow convergence and suboptimal rate-distortion performance. In this work, we demonstrate that a simple utilization of a second-order quasi-Newton optimizer, \textbf{SOAP}, dramatically improves both training efficiency and final performance across diverse LICs. Our theoretical and empirical analyses reveal that Newton preconditioning inherently resolves the intra-step and inter-step update conflicts intrinsic to the R-D objective, facilitating faster, more stable convergence. Beyond acceleration, we uncover a critical deployability benefit: second-order trained models exhibit significantly fewer activation and latent outliers. This substantially enhances robustness to post-training quantization. Together, these results establish second-order optimization, achievable as a seamless drop-in replacement of the imported optimizer, as a powerful, practical tool for advancing the efficiency and real-world readiness of LICs.