Adaptive Dense-to-Sparse Paradigm for Pruning Online Recommendation System with Non-Stationary Data
This work addresses the problem of reducing computational and memory costs for large-scale recommendation systems in data centers, which is incremental as it adapts existing pruning techniques to handle non-stationary data.
The paper tackles the challenge of pruning online recommendation systems with non-stationary data, where traditional methods cause accuracy loss, and presents an adaptive dense-to-sparse paradigm with a novel pruning algorithm that automatically learns sparsity patterns across layers to maintain performance.
Large scale deep learning provides a tremendous opportunity to improve the quality of content recommendation systems by employing both wider and deeper models, but this comes at great infrastructural cost and carbon footprint in modern data centers. Pruning is an effective technique that reduces both memory and compute demand for model inference. However, pruning for online recommendation systems is challenging due to the continuous data distribution shift (a.k.a non-stationary data). Although incremental training on the full model is able to adapt to the non-stationary data, directly applying it on the pruned model leads to accuracy loss. This is because the sparsity pattern after pruning requires adjustment to learn new patterns. To the best of our knowledge, this is the first work to provide in-depth analysis and discussion of applying pruning to online recommendation systems with non-stationary data distribution. Overall, this work makes the following contributions: 1) We present an adaptive dense to sparse paradigm equipped with a novel pruning algorithm for pruning a large scale recommendation system with non-stationary data distribution; 2) We design the pruning algorithm to automatically learn the sparsity across layers to avoid repeating hand-tuning, which is critical for pruning the heterogeneous architectures of recommendation systems trained with non-stationary data.