Memory-efficient Embedding for Recommendations
This addresses memory constraints in practical recommender systems for companies handling thousands of features, though it is incremental as it builds on existing AutoML and embedding techniques.
The paper tackles the memory inefficiency of using a unified embedding dimension for all feature fields in large-scale recommender systems by proposing AutoDim, an AutoML-based framework that automatically assigns different embedding dimensions to feature fields based on their importance, achieving up to 90% memory reduction with comparable or improved accuracy.
Practical large-scale recommender systems usually contain thousands of feature fields from users, items, contextual information, and their interactions. Most of them empirically allocate a unified dimension to all feature fields, which is memory inefficient. Thus it is highly desired to assign different embedding dimensions to different feature fields according to their importance and predictability. Due to the large amounts of feature fields and the nuanced relationship between embedding dimensions with feature distributions and neural network architectures, manually allocating embedding dimensions in practical recommender systems can be very difficult. To this end, we propose an AutoML based framework (AutoDim) in this paper, which can automatically select dimensions for different feature fields in a data-driven fashion. Specifically, we first proposed an end-to-end differentiable framework that can calculate the weights over various dimensions for feature fields in a soft and continuous manner with an AutoML based optimization algorithm; then we derive a hard and discrete embedding component architecture according to the maximal weights and retrain the whole recommender framework. We conduct extensive experiments on benchmark datasets to validate the effectiveness of the AutoDim framework.