LEARN: An Invex Loss for Outlier Oblivious Robust Online Optimization
This work addresses robust optimization for online learning systems under outlier corruption, offering a novel method but with incremental improvements in a specialized domain.
The paper tackles robust online convex optimization with adversarial outliers by introducing the LEARN loss, a non-convex (invex) function, and achieves tight regret guarantees in a dynamic setting without Lipschitz assumptions.
We study a robust online convex optimization framework, where an adversary can introduce outliers by corrupting loss functions in an arbitrary number of rounds k, unknown to the learner. Our focus is on a novel setting allowing unbounded domains and large gradients for the losses without relying on a Lipschitz assumption. We introduce the Log Exponential Adjusted Robust and iNvex (LEARN) loss, a non-convex (invex) robust loss function to mitigate the effects of outliers and develop a robust variant of the online gradient descent algorithm by leveraging the LEARN loss. We establish tight regret guarantees (up to constants), in a dynamic setting, with respect to the uncorrupted rounds and conduct experiments to validate our theory. Furthermore, we present a unified analysis framework for developing online optimization algorithms for non-convex (invex) losses, utilizing it to provide regret bounds with respect to the LEARN loss, which may be of independent interest.