DimGrow: Memory-Efficient Field-level Embedding Dimension Search
This addresses memory efficiency challenges in large-scale recommendation systems, offering a lightweight solution for automated dimension search, though it appears incremental as it builds on prior work like pruning and NAS.
The paper tackles the problem of automated embedding dimension allocation for key feature fields in recommendation systems, proposing DimGrow to eliminate the memory-intensive SuperNet requirement and achieving memory-efficient training with reduced memory usage compared to existing methods.
Key feature fields need bigger embedding dimensionality, others need smaller. This demands automated dimension allocation. Existing approaches, such as pruning or Neural Architecture Search (NAS), require training a memory-intensive SuperNet that enumerates all possible dimension combinations, which is infeasible for large feature spaces. We propose DimGrow, a lightweight approach that eliminates the SuperNet requirement. Starting training model from one dimension per feature field, DimGrow can progressively expand/shrink dimensions via importance scoring. Dimensions grow only when their importance consistently exceed a threshold, ensuring memory efficiency. Experiments on three recommendation datasets verify the effectiveness of DimGrow while it reduces training memory compared to SuperNet-based methods.