Benefits of Online Tilted Empirical Risk Minimization: A Case Study of Outlier Detection and Robust Regression
This work addresses the problem of balancing robustness and fairness in streaming data for machine learning practitioners, though it is incremental as it adapts an existing method to online scenarios.
The paper tackled the limitation of Tilted Empirical Risk Minimization (TERM) in online settings, where it degenerates to standard ERM, by proposing an online TERM formulation that preserves tilt effects without extra overhead. The results showed that negative tilting suppressed outlier influence and positive tilting improved recall with minimal precision impact, all at per-sample computational cost equivalent to ERM.
Empirical Risk Minimization (ERM) is a foundational framework for supervised learning but primarily optimizes average-case performance, often neglecting fairness and robustness considerations. Tilted Empirical Risk Minimization (TERM) extends ERM by introducing an exponential tilt hyperparameter $t$ to balance average-case accuracy with worst-case fairness and robustness. However, in online or streaming settings where data arrive one sample at a time, the classical TERM objective degenerates to standard ERM, losing tilt sensitivity. We address this limitation by proposing an online TERM formulation that removes the logarithm from the classical objective, preserving tilt effects without additional computational or memory overhead. This formulation enables a continuous trade-off controlled by $t$, smoothly interpolating between ERM ($t \to 0$), fairness emphasis ($t > 0$), and robustness to outliers ($t < 0$). We empirically validate online TERM on two representative streaming tasks: robust linear regression with adversarial outliers and minority-class detection in binary classification. Our results demonstrate that negative tilting effectively suppresses outlier influence, while positive tilting improves recall with minimal impact on precision, all at per-sample computational cost equivalent to ERM. Online TERM thus recovers the full robustness-fairness spectrum of classical TERM in an efficient single-sample learning regime.