Bandits with Partially Observable Confounded Data
This addresses the challenge of efficiently using confounded offline data to enhance online bandit algorithms, which is an incremental improvement in the field of reinforcement learning.
The paper tackles the problem of linear contextual bandits with access to confounded offline data, showing that such data can be leveraged to improve online learning algorithms. The results include regret bounds that improve current bounds by a factor related to the visible dimensionality of contexts in the data.
We study linear contextual bandits with access to a large, confounded, offline dataset that was sampled from some fixed policy. We show that this problem is closely related to a variant of the bandit problem with side information. We construct a linear bandit algorithm that takes advantage of the projected information, and prove regret bounds. Our results demonstrate the ability to take advantage of confounded offline data. Particularly, we prove regret bounds that improve current bounds by a factor related to the visible dimensionality of the contexts in the data. Our results indicate that confounded offline data can significantly improve online learning algorithms. Finally, we demonstrate various characteristics of our approach through synthetic simulations.