Gaussian Imagination in Bandit Learning
This formalizes the folklore that Bayesian agents remain effective with diffuse misspecified distributions, which is incremental as it builds on existing theory without introducing new methods.
The paper tackles the problem of Bayesian agents using misspecified Gaussian distributions in Bernoulli bandits, showing that if the Gaussian prior and likelihood are sufficiently diffuse, the increase in regret grows at most linearly with the square-root of the time horizon, making the per-timestep increase vanish.
Assuming distributions are Gaussian often facilitates computations that are otherwise intractable. We study the performance of an agent that attains a bounded information ratio with respect to a bandit environment with a Gaussian prior distribution and a Gaussian likelihood function when applied instead to a Bernoulli bandit. Relative to an information-theoretic bound on the Bayesian regret the agent would incur when interacting with the Gaussian bandit, we bound the increase in regret when the agent interacts with the Bernoulli bandit. If the Gaussian prior distribution and likelihood function are sufficiently diffuse, this increase grows at a rate which is at most linear in the square-root of the time horizon, and thus the per-timestep increase vanishes. Our results formalize the folklore that so-called Bayesian agents remain effective when instantiated with diffuse misspecified distributions.