Online learning with graph-structured feedback against adaptive adversaries
This work addresses theoretical guarantees for online learning algorithms in adversarial environments with structured feedback, which is incremental to existing bounds.
The paper tackles the problem of online learning with graph-structured feedback against adaptive adversaries, deriving upper bounds of O~(T^{2/3}) and O~(T^{3/4}) for strongly- and weakly-observable graphs, and matching lower bounds of Ω~(T^{2/3}) in specific cases.
We derive upper and lower bounds for the policy regret of $T$-round online learning problems with graph-structured feedback, where the adversary is nonoblivious but assumed to have a bounded memory. We obtain upper bounds of $\widetilde O(T^{2/3})$ and $\widetilde O(T^{3/4})$ for strongly-observable and weakly-observable graphs, respectively, based on analyzing a variant of the Exp3 algorithm. When the adversary is allowed a bounded memory of size 1, we show that a matching lower bound of $\widetildeΩ(T^{2/3})$ is achieved in the case of full-information feedback. We also study the particular loss structure of an oblivious adversary with switching costs, and show that in such a setting, non-revealing strongly-observable feedback graphs achieve a lower bound of $\widetildeΩ(T^{2/3})$, as well.