Learning to Collaborate in Multi-Module Recommendation via Multi-Agent Reinforcement Learning without Communication
This addresses the issue of inefficient recommendation systems for online platforms, though it is incremental as it builds on existing multi-agent and game theory concepts.
The paper tackles the problem of sub-optimal global recommendations on e-commerce web pages due to independent modules competing without cooperation, proposing a multi-agent reinforcement learning approach that achieves superior performance over baselines in experiments using real-world data.
With the rise of online e-commerce platforms, more and more customers prefer to shop online. To sell more products, online platforms introduce various modules to recommend items with different properties such as huge discounts. A web page often consists of different independent modules. The ranking policies of these modules are decided by different teams and optimized individually without cooperation, which might result in competition between modules. Thus, the global policy of the whole page could be sub-optimal. In this paper, we propose a novel multi-agent cooperative reinforcement learning approach with the restriction that different modules cannot communicate. Our contributions are three-fold. Firstly, inspired by a solution concept in game theory named correlated equilibrium, we design a signal network to promote cooperation of all modules by generating signals (vectors) for different modules. Secondly, an entropy-regularized version of the signal network is proposed to coordinate agents' exploration of the optimal global policy. Furthermore, experiments based on real-world e-commerce data demonstrate that our algorithm obtains superior performance over baselines.