The Combinatorial Multi-Bandit Problem and its Application to Energy Management
This addresses energy systems management by providing a novel framework for optimizing combinatorial objectives with multiple observable bandits, though it appears incremental as it generalizes existing bandit theory.
The paper tackles the Combinatorial Multi-Bandit Problem, motivated by energy management, by developing algorithms that combine multi-arm bandit exploration with mathematical programming, achieving effective learning for 150 bandits with 24 actions each over 365 episodes.
We study a Combinatorial Multi-Bandit Problem motivated by applications in energy systems management. Given multiple probabilistic multi-arm bandits with unknown outcome distributions, the task is to optimize the value of a combinatorial objective function mapping the vector of individual bandit outcomes to a single scalar reward. Unlike in single-bandit problems with multi-dimensional action space, the outcomes of the individual bandits are observable in our setting and the objective function is known. Guided by the hypothesis that individual observability enables better trade-offs between exploration and exploitation, we generalize the lower regret bound for single bandits, showing that indeed for multiple bandits it admits parallelized exploration. For our energy management application we propose a range of algorithms that combine exploration principles for multi-arm bandits with mathematical programming. In an experimental study we demonstrate the effectiveness of our approach to learn action assignments for 150 bandits, each having 24 actions, within a horizon of 365 episodes.