Adopting Robustness and Optimality in Fitting and Learning
This work addresses robustness and optimality issues in machine learning fitting tasks, but it appears incremental as it builds on existing estimators.
The authors tackled the problem of achieving robustness to outliers and avoiding local optima in fitting and learning tasks by generalizing a modified exponentialized estimator with a robust-optimal index. The results showed effectiveness in experiments on noisy non-convex functions and the MNIST dataset, though no concrete numbers were provided.
We generalized a modified exponentialized estimator by pushing the robust-optimal (RO) index $λ$ to $-\infty$ for achieving robustness to outliers by optimizing a quasi-Minimin function. The robustness is realized and controlled adaptively by the RO index without any predefined threshold. Optimality is guaranteed by expansion of the convexity region in the Hessian matrix to largely avoid local optima. Detailed quantitative analysis on both robustness and optimality are provided. The results of proposed experiments on fitting tasks for three noisy non-convex functions and the digits recognition task on the MNIST dataset consolidate the conclusions.