When can we improve on sample average approximation for stochastic optimization?
This work addresses the problem of method selection in stochastic optimization for researchers and practitioners, but it is incremental as it evaluates existing methods without introducing new ones.
The paper compared sample average approximation with bagging, kernel smoothing, maximum likelihood estimation, and a Bayesian approach for stochastic optimization when distribution information is available, finding that sample average approximation was effective in a quadratic test set but outperformed by Bayesian methods, while bagging, MLE, and Bayesian approaches performed well in a portfolio optimization test set.
We explore the performance of sample average approximation in comparison with several other methods for stochastic optimization when there is information available on the underlying true probability distribution. The methods we evaluate are (a) bagging; (b) kernel smoothing; (c) maximum likelihood estimation (MLE); and (d) a Bayesian approach. We use two test sets, the first has a quadratic objective function allowing for very different types of interaction between the random component and the univariate decision variable. Here the sample average approximation is remarkably effective and only consistently outperformed by a Bayesian approach. The second test set is a portfolio optimization problem in which we use different covariance structures for a set of 5 stocks. Here bagging, MLE and a Bayesian approach all do well.