Deep Residual Learning in the JPEG Transform Domain
This work addresses the need for efficient image processing in domains like mobile or embedded systems by reducing computational overhead without sacrificing accuracy, though it is incremental as it adapts existing methods to a compressed format.
The authors tackled the problem of performing deep residual network inference and learning directly on JPEG-compressed images, resulting in a method that maintains mathematical equivalence to spatial domain networks with minimal accuracy loss while enabling faster processing due to JPEG sparsity.
We introduce a general method of performing Residual Network inference and learning in the JPEG transform domain that allows the network to consume compressed images as input. Our formulation leverages the linearity of the JPEG transform to redefine convolution and batch normalization with a tune-able numerical approximation for ReLu. The result is mathematically equivalent to the spatial domain network up to the ReLu approximation accuracy. A formulation for image classification and a model conversion algorithm for spatial domain networks are given as examples of the method. We show that the sparsity of the JPEG format allows for faster processing of images with little to no penalty in the network accuracy.