Combining Offline Causal Inference and Online Bandit Learning for Data Driven Decision
This addresses a fundamental challenge for companies with large datasets in making adaptive decisions to avoid continuous user experience damage, representing an incremental improvement by combining existing techniques.
The paper tackles the problem of using both logged historical data and incoming streaming data to improve decision-making, proposing a framework that unifies offline causal inference and online bandit learning, with experiments on real datasets showing it outperforms methods using only one data source or improper data handling.
A fundamental question for companies with large amount of logged data is: How to use such logged data together with incoming streaming data to make good decisions? Many companies currently make decisions via online A/B tests, but wrong decisions during testing hurt users' experiences and cause irreversible damage. A typical alternative is offline causal inference, which analyzes logged data alone to make decisions. However, these decisions are not adaptive to the new incoming data, and so a wrong decision will continuously hurt users' experiences. To overcome the aforementioned limitations, we propose a framework to unify offline causal inference algorithms (e.g., weighting, matching) and online learning algorithms (e.g., UCB, LinUCB). We propose novel algorithms and derive bounds on the decision accuracy via the notion of "regret". We derive the first upper regret bound for forest-based online bandit algorithms. Experiments on two real datasets show that our algorithms outperform other algorithms that use only logged data or online feedbacks, or algorithms that do not use the data properly.