Online Learning and Decision-Making under Generalized Linear Model with High-Dimensional Data
This work addresses decision-making in high-dimensional online settings, such as personalized medicine or advertising, with incremental improvements in regret bounds and sparsity handling.
The paper tackles online decision-making with high-dimensional data by proposing the G-MCP-Bandit algorithm, which achieves optimal cumulative regret bounds of O(log T) and O(log d) and outperforms benchmarks in experiments with synthetic and real datasets under conditions like high sparsity.
We propose a minimax concave penalized multi-armed bandit algorithm under generalized linear model (G-MCP-Bandit) for a decision-maker facing high-dimensional data in an online learning and decision-making process. We demonstrate that the G-MCP-Bandit algorithm asymptotically achieves the optimal cumulative regret in the sample size dimension T , O(log T), and further attains a tight bound in the covariate dimension d, O(log d). In addition, we develop a linear approximation method, the 2-step weighted Lasso procedure, to identify the MCP estimator for the G-MCP-Bandit algorithm under non-iid samples. Under this procedure, the MCP estimator matches the oracle estimator with high probability and converges to the true parameters with the optimal convergence rate. Finally, through experiments based on synthetic data and two real datasets (warfarin dosing dataset and Tencent search advertising dataset), we show that the G-MCP-Bandit algorithm outperforms other benchmark algorithms, especially when there is a high level of data sparsity or the decision set is large.