Adaptive Oracle-Efficient Online Learning
This work addresses a computational limitation in online learning for practitioners in fields like auctions and classification, though it is incremental as it builds on existing oracle-efficient methods.
The paper tackles the problem of oracle-efficient online learning algorithms not adapting well to friendly environments like small-loss problems and IID data, and introduces a new framework for follow-the-perturbed-leader algorithms that achieve oracle-efficiency and adapt to these scenarios, with applications in online auctions and transductive online classification.
The classical algorithms for online learning and decision-making have the benefit of achieving the optimal performance guarantees, but suffer from computational complexity limitations when implemented at scale. More recent sophisticated techniques, which we refer to as oracle-efficient methods, address this problem by dispatching to an offline optimization oracle that can search through an exponentially-large (or even infinite) space of decisions and select that which performed the best on any dataset. But despite the benefits of computational feasibility, oracle-efficient algorithms exhibit one major limitation: while performing well in worst-case settings, they do not adapt well to friendly environments. In this paper we consider two such friendly scenarios, (a) "small-loss" problems and (b) IID data. We provide a new framework for designing follow-the-perturbed-leader algorithms that are oracle-efficient and adapt well to the small-loss environment, under a particular condition which we call approximability (which is spiritually related to sufficient conditions provided by Dudík et al., [2020]). We identify a series of real-world settings, including online auctions and transductive online classification, for which approximability holds. We also extend the algorithm to an IID data setting and establish a "best-of-both-worlds" bound in the oracle-efficient setting.