Ads Recommendation in a Collapsed and Entangled World
This work addresses feature representation issues in large-scale ads recommendation systems for platforms like Tencent, though it is incremental, building on over a decade of team efforts.
The paper tackles the challenges of dimensional collapse and interest entanglement in ads recommendation representations, proposing practical approaches and analysis tools that improve robustness and disentanglement, with performance validated on a platform handling hundreds of billions of daily requests.
We present Tencent's ads recommendation system and examine the challenges and practices of learning appropriate recommendation representations. Our study begins by showcasing our approaches to preserving prior knowledge when encoding features of diverse types into embedding representations. We specifically address sequence features, numeric features, and pre-trained embedding features. Subsequently, we delve into two crucial challenges related to feature representation: the dimensional collapse of embeddings and the interest entanglement across different tasks or scenarios. We propose several practical approaches to address these challenges that result in robust and disentangled recommendation representations. We then explore several training techniques to facilitate model optimization, reduce bias, and enhance exploration. Additionally, we introduce three analysis tools that enable us to study feature correlation, dimensional collapse, and interest entanglement. This work builds upon the continuous efforts of Tencent's ads recommendation team over the past decade. It summarizes general design principles and presents a series of readily applicable solutions and analysis tools. The reported performance is based on our online advertising platform, which handles hundreds of billions of requests daily and serves millions of ads to billions of users.