Boosted Off-Policy Learning
This work addresses the challenge of efficient policy optimization in bandit settings for machine learning practitioners, offering a novel method that leverages existing supervised learning tools.
The authors tackled the problem of off-policy learning from logged bandit feedback by proposing the first boosting algorithm that directly optimizes an estimate of a policy's expected reward, and experiments showed it can outperform methods like deep neural networks and reward regression.
We propose the first boosting algorithm for off-policy learning from logged bandit feedback. Unlike existing boosting methods for supervised learning, our algorithm directly optimizes an estimate of the policy's expected reward. We analyze this algorithm and prove that the excess empirical risk decreases (possibly exponentially fast) with each round of boosting, provided a ''weak'' learning condition is satisfied by the base learner. We further show how to reduce the base learner to supervised learning, which opens up a broad range of readily available base learners with practical benefits, such as decision trees. Experiments indicate that our algorithm inherits many desirable properties of tree-based boosting algorithms (e.g., robustness to feature scaling and hyperparameter tuning), and that it can outperform off-policy learning with deep neural networks as well as methods that simply regress on the observed rewards.