Stay With Me: Lifetime Maximization Through Heteroscedastic Linear Bandits With Reneging
This addresses the challenge of participant retention in decision-making systems, but it is incremental as it builds on existing bandit frameworks with specific extensions.
The paper tackles the problem of sequential decision-making for lifetime maximization in applications like medical treatment and portfolio selection, where participants may disengage after unsatisfying outcomes, by proposing a heteroscedastic linear bandits model with reneging and developing an HR-UCB policy that achieves O(√T(log T)^3) regret, validated through simulations.
Sequential decision making for lifetime maximization is a critical problem in many real-world applications, such as medical treatment and portfolio selection. In these applications, a "reneging" phenomenon, where participants may disengage from future interactions after observing an unsatisfiable outcome, is rather prevalent. To address the above issue, this paper proposes a model of heteroscedastic linear bandits with reneging, which allows each participant to have a distinct "satisfaction level," with any interaction outcome falling short of that level resulting in that participant reneging. Moreover, it allows the variance of the outcome to be context-dependent. Based on this model, we develop a UCB-type policy, namely HR-UCB, and prove that it achieves $\mathcal{O}\big(\sqrt{{T}(\log({T}))^{3}}\big)$ regret. Finally, we validate the performance of HR-UCB via simulations.