Hierarchical Reinforcement Learning for Temporal Abstraction of Listwise Recommendation
This work provides a novel approach for improving recommendation systems by handling temporal abstraction, though it appears incremental as it builds on existing hierarchical reinforcement learning methods.
The paper tackled the problem of listwise recommendation by addressing long-term user perceptions and short-term interest shifts using a hierarchical reinforcement learning framework, resulting in significant performance improvements over baselines in both simulator-based and industrial dataset experiments.
Modern listwise recommendation systems need to consider both long-term user perceptions and short-term interest shifts. Reinforcement learning can be applied on recommendation to study such a problem but is also subject to large search space, sparse user feedback and long interactive latency. Motivated by recent progress in hierarchical reinforcement learning, we propose a novel framework called mccHRL to provide different levels of temporal abstraction on listwise recommendation. Within the hierarchical framework, the high-level agent studies the evolution of user perception, while the low-level agent produces the item selection policy by modeling the process as a sequential decision-making problem. We argue that such framework has a well-defined decomposition of the outra-session context and the intra-session context, which are encoded by the high-level and low-level agents, respectively. To verify this argument, we implement both a simulator-based environment and an industrial dataset-based experiment. Results observe significant performance improvement by our method, compared with several well-known baselines. Data and codes have been made public.