From Features to Transformers: Redefining Ranking for Scalable Impact
This work addresses scalable ranking for platforms like LinkedIn, offering a novel method that deprecates extensive feature engineering and enhances diversity, though it builds incrementally on existing transformer architectures.
The paper tackles the problem of large-scale ranking for recommendation systems by introducing LiGR, a transformer-based framework that reduces manual feature engineering from hundreds to few features, achieving state-of-the-art performance and validating scaling laws with improved metrics from larger models and data.
We present LiGR, a large-scale ranking framework developed at LinkedIn that brings state-of-the-art transformer-based modeling architectures into production. We introduce a modified transformer architecture that incorporates learned normalization and simultaneous set-wise attention to user history and ranked items. This architecture enables several breakthrough achievements, including: (1) the deprecation of most manually designed feature engineering, outperforming the prior state-of-the-art system using only few features (compared to hundreds in the baseline), (2) validation of the scaling law for ranking systems, showing improved performance with larger models, more training data, and longer context sequences, and (3) simultaneous joint scoring of items in a set-wise manner, leading to automated improvements in diversity. To enable efficient serving of large ranking models, we describe techniques to scale inference effectively using single-pass processing of user history and set-wise attention. We also summarize key insights from various ablation studies and A/B tests, highlighting the most impactful technical approaches.