Reinforcement Learning-based Product Delivery Frequency Control
This work addresses the problem of optimizing product delivery frequency in large-scale recommender systems to balance daily metrics and resource consumption for businesses.
The authors propose a personalized methodology for controlling the frequency of product deliveries in recommender systems, combining reinforcement learning for long-term value optimization with an "Effective Factor" for robust volume control. Their method achieved statistically significant improvements in daily metrics and resource efficiency across several notification applications serving billions of users.
Frequency control is an important problem in modern recommender systems. It dictates the delivery frequency of recommendations to maintain product quality and efficiency. For example, the frequency of delivering promotional notifications impacts daily metrics as well as the infrastructure resource consumption (e.g. CPU and memory usage). There remain open questions on what objective we should optimize to represent business values in the long term best, and how we should balance between daily metrics and resource consumption in a dynamically fluctuating environment. We propose a personalized methodology for the frequency control problem, which combines long-term value optimization using reinforcement learning (RL) with a robust volume control technique we termed "Effective Factor". We demonstrate statistically significant improvement in daily metrics and resource efficiency by our method in several notification applications at a scale of billions of users. To our best knowledge, our study represents the first deep RL application on the frequency control problem at such an industrial scale.