AIF: Asynchronous Inference Framework for Cost-Effective Pre-Ranking
This addresses efficiency and latency issues in large-scale recommendation systems for industrial applications like e-commerce, though it appears incremental as it builds on existing pre-ranking frameworks.
The paper tackles bottlenecks in industrial recommendation systems by proposing the Asynchronous Inference Framework (AIF), which decouples interaction-independent computations to run in parallel with retrieval, resulting in enhanced computational efficiency, reduced latency, and successful deployment in Taobao's advertising system.
In industrial recommendation systems, pre-ranking models based on deep neural networks (DNNs) commonly adopt a sequential execution framework: feature fetching and model forward computation are triggered only after receiving candidates from the upstream retrieval stage. This design introduces inherent bottlenecks, including redundant computations of identical users/items and increased latency due to strictly sequential operations, which jointly constrain the model's capacity and system efficiency. To address these limitations, we propose the Asynchronous Inference Framework (AIF), a cost-effective computational architecture that decouples interaction-independent components, those operating within a single user or item, from real-time prediction. AIF reorganizes the model inference process by performing user-side computations in parallel with the retrieval stage and conducting item-side computations in a nearline manner. This means that interaction-independent components are calculated just once and completed before the real-time prediction phase of the pre-ranking stage. As a result, AIF enhances computational efficiency and reduces latency, freeing up resources to significantly improve the feature set and model architecture of interaction-independent components. Moreover, we delve into model design within the AIF framework, employing approximated methods for interaction-dependent components in online real-time predictions. By co-designing both the framework and the model, our solution achieves notable performance gains without significantly increasing computational and latency costs. This has enabled the successful deployment of AIF in the Taobao display advertising system.