Understanding the Eluder Dimension
This work provides theoretical insights for researchers in online learning and reinforcement learning, but it is incremental as it builds on existing complexity measures without introducing new algorithms or broad applications.
The paper investigates the eluder dimension, a complexity measure used in online bandits and reinforcement learning, by analyzing its relationship with generalized rank and showing that eluder dimension can be exponentially smaller or larger than σ-rank depending on activation function properties, and characterizes it for binary-valued classes in terms of star number and threshold dimension.
We provide new insights on eluder dimension, a complexity measure that has been extensively used to bound the regret of algorithms for online bandits and reinforcement learning with function approximation. First, we study the relationship between the eluder dimension for a function class and a generalized notion of rank, defined for any monotone "activation" $σ: \mathbb{R}\to \mathbb{R}$, which corresponds to the minimal dimension required to represent the class as a generalized linear model. It is known that when $σ$ has derivatives bounded away from $0$, $σ$-rank gives rise to an upper bound on eluder dimension for any function class; we show however that eluder dimension can be exponentially smaller than $σ$-rank. We also show that the condition on the derivative is necessary; namely, when $σ$ is the $\mathsf{relu}$ activation, the eluder dimension can be exponentially larger than $σ$-rank. For binary-valued function classes, we obtain a characterization of the eluder dimension in terms of star number and threshold dimension, quantities which are relevant in active learning and online learning respectively.