Enhancing Taobao Display Advertising with Multimodal Representations: Challenges, Approaches and Insights
This addresses the problem of enhancing industrial-scale recommendation systems for advertisers and platforms, though it is incremental as it builds on existing ID-based models.
The paper tackled the challenge of integrating multimodal data into Taobao's display advertising system to improve recommendation accuracy, resulting in significant performance improvements observed since mid-2023.
Despite the recognized potential of multimodal data to improve model accuracy, many large-scale industrial recommendation systems, including Taobao display advertising system, predominantly depend on sparse ID features in their models. In this work, we explore approaches to leverage multimodal data to enhance the recommendation accuracy. We start from identifying the key challenges in adopting multimodal data in a manner that is both effective and cost-efficient for industrial systems. To address these challenges, we introduce a two-phase framework, including: 1) the pre-training of multimodal representations to capture semantic similarity, and 2) the integration of these representations with existing ID-based models. Furthermore, we detail the architecture of our production system, which is designed to facilitate the deployment of multimodal representations. Since the integration of multimodal representations in mid-2023, we have observed significant performance improvements in Taobao display advertising system. We believe that the insights we have gathered will serve as a valuable resource for practitioners seeking to leverage multimodal data in their systems.