Optimistic Robust Optimization With Applications To Machine Learning
This work offers a novel perspective on uncertainty in optimization, potentially benefiting machine learning practitioners by linking existing methods to a unified framework, though it appears incremental in its reinterpretation of known techniques.
The paper introduces an optimistic robust optimization approach as an alternative to traditional worst-case methods, applying it to reduce conservatism in robust linear programming and providing new interpretations for sparsity-inducing regularization and outlier-handling techniques in machine learning.
Robust Optimization has traditionally taken a pessimistic, or worst-case viewpoint of uncertainty which is motivated by a desire to find sets of optimal policies that maintain feasibility under a variety of operating conditions. In this paper, we explore an optimistic, or best-case view of uncertainty and show that it can be a fruitful approach. We show that these techniques can be used to address a wide variety of problems. First, we apply our methods in the context of robust linear programming, providing a method for reducing conservatism in intuitive ways that encode economically realistic modeling assumptions. Second, we look at problems in machine learning and find that this approach is strongly connected to the existing literature. Specifically, we provide a new interpretation for popular sparsity inducing non-convex regularization schemes. Additionally, we show that successful approaches for dealing with outliers and noise can be interpreted as optimistic robust optimization problems. Although many of the problems resulting from our approach are non-convex, we find that DCA or DCA-like optimization approaches can be intuitive and efficient.