Thompson Sampling for Noncompliant Bandits
This work addresses a specific limitation in bandit algorithms for scenarios like clinical trials or online advertising where agents may not comply with chosen actions, offering incremental improvements over existing methods.
The paper tackles the problem of noncompliance in bandit problems, where traditional Thompson sampling assumes perfect action implementation, by introducing a stochastic noncompliance model and deriving variants that handle observed and latent noncompliance, resulting in algorithms that match or outperform traditional methods in both compliant and noncompliant environments.
Thompson sampling, a Bayesian method for balancing exploration and exploitation in bandit problems, has theoretical guarantees and exhibits strong empirical performance in many domains. Traditional Thompson sampling, however, assumes perfect compliance, where an agent's chosen action is treated as the implemented action. This article introduces a stochastic noncompliance model that relaxes this assumption. We prove that any noncompliance in a 2-armed Bernoulli bandit increases existing regret bounds. With our noncompliance model, we derive Thompson sampling variants that explicitly handle both observed and latent noncompliance. With extensive empirical analysis, we demonstrate that our algorithms either match or outperform traditional Thompson sampling in both compliant and noncompliant environments.