Nearly Dimension-Independent Sparse Linear Bandit over Small Action Spaces via Best Subset Selection
This addresses the exploration-exploitation challenge in personalized recommendation, online advertising, and personalized medicine, representing a strong theoretical advance with practical implementation.
The paper tackles the high-dimensional stochastic contextual bandit problem with finite random action spaces, proposing an algorithm using doubly growing epochs and best subset selection that achieves $ ilde{\mathcal{O}}(s\sqrt{T})$ regret nearly independent of ambient dimension $d$, and further attains $ ilde{\mathcal{O}}(\sqrt{sT})$ regret matching the minimax lower bound.
We consider the stochastic contextual bandit problem under the high dimensional linear model. We focus on the case where the action space is finite and random, with each action associated with a randomly generated contextual covariate. This setting finds essential applications such as personalized recommendation, online advertisement, and personalized medicine. However, it is very challenging as we need to balance exploration and exploitation. We propose doubly growing epochs and estimating the parameter using the best subset selection method, which is easy to implement in practice. This approach achieves $ \tilde{\mathcal{O}}(s\sqrt{T})$ regret with high probability, which is nearly independent in the ``ambient'' regression model dimension $d$. We further attain a sharper $\tilde{\mathcal{O}}(\sqrt{sT})$ regret by using the \textsc{SupLinUCB} framework and match the minimax lower bound of low-dimensional linear stochastic bandit problems. Finally, we conduct extensive numerical experiments to demonstrate the applicability and robustness of our algorithms empirically.