Hierarchical Causal Bandit
This work addresses a gap in causal bandit theory for researchers, though it is incremental as it builds on existing models by extending to dependent variables.
The paper tackles the problem of causal bandits with dependent variables, which previous work had only addressed for independent variables, and introduces a hierarchical model that yields nearly matching regret bounds for binary contexts.
Causal bandit is a nascent learning model where an agent sequentially experiments in a causal network of variables, in order to identify the reward-maximizing intervention. Despite the model's wide applicability, existing analytical results are largely restricted to a parallel bandit version where all variables are mutually independent. We introduce in this work the hierarchical causal bandit model as a viable path towards understanding general causal bandits with dependent variables. The core idea is to incorporate a contextual variable that captures the interaction among all variables with direct effects. Using this hierarchical framework, we derive sharp insights on algorithmic design in causal bandits with dependent arms and obtain nearly matching regret bounds in the case of a binary context.