Diversity-Driven Selection of Exploration Strategies in Multi-Armed Bandits
This addresses the challenge of adaptive strategy selection for agents in robotics or AI, but it is incremental as it builds on existing bandit methods with a new reward metric.
The paper tackles the problem of selecting exploration strategies in unknown environments by proposing a strategy-agnostic method that uses diversity of effects as a reward signal in a Multi-Armed Bandits framework, and shows it can discriminate between strategies with tenuous differences and achieve competitive performance with the best fixed mixture in a simulated robotic arm.
We consider a scenario where an agent has multiple available strategies to explore an unknown environment. For each new interaction with the environment, the agent must select which exploration strategy to use. We provide a new strategy-agnostic method that treat the situation as a Multi-Armed Bandits problem where the reward signal is the diversity of effects that each strategy produces. We test the method empirically on a simulated planar robotic arm, and establish that the method is both able discriminate between strategies of dissimilar quality, even when the differences are tenuous, and that the resulting performance is competitive with the best fixed mixture of strategies.