LCS: Learning Compressible Subspaces for Adaptive Network Compression at Inference Time
This enables adaptive network compression on-device for real-world systems with unstable resource guarantees, representing an incremental improvement over existing subspace methods.
The paper tackles the problem of adapting neural network compression to varying computational resources at inference time by training a compressible subspace of models that range from efficient to accurate, achieving high accuracy for sparsity rates above 90% and on-par accuracy for quantization at variable bit widths.
When deploying deep learning models to a device, it is traditionally assumed that available computational resources (compute, memory, and power) remain static. However, real-world computing systems do not always provide stable resource guarantees. Computational resources need to be conserved when load from other processes is high or battery power is low. Inspired by recent works on neural network subspaces, we propose a method for training a "compressible subspace" of neural networks that contains a fine-grained spectrum of models that range from highly efficient to highly accurate. Our models require no retraining, thus our subspace of models can be deployed entirely on-device to allow adaptive network compression at inference time. We present results for achieving arbitrarily fine-grained accuracy-efficiency trade-offs at inference time for structured and unstructured sparsity. We achieve accuracies on-par with standard models when testing our uncompressed models, and maintain high accuracy for sparsity rates above 90% when testing our compressed models. We also demonstrate that our algorithm extends to quantization at variable bit widths, achieving accuracy on par with individually trained networks.