Non-Convex Optimizations for Machine Learning with Theoretical Guarantee: Robust Matrix Completion and Neural Network Learning
It addresses the need for explainable learning systems to enhance safety and privacy in machine learning applications, though it appears incremental as it builds on existing non-convex optimization frameworks.
The thesis tackles the challenge of providing explainable algorithms for non-convex optimization problems in machine learning, focusing on low-rank matrix completion and neural network learning, and aims to develop methods with theoretical guarantees.
Despite the recent development in machine learning, most learning systems are still under the concept of "black box", where the performance cannot be understood and derived. With the rise of safety and privacy concerns in public, designing an explainable learning system has become a new trend in machine learning. In general, many machine learning problems are formulated as minimizing (or maximizing) some loss function. Since real data are most likely generated from non-linear models, the loss function is non-convex in general. Unlike the convex optimization problem, gradient descent algorithms will be trapped in spurious local minima in solving non-convex optimization. Therefore, it is challenging to provide explainable algorithms when studying non-convex optimization problems. In this thesis, two popular non-convex problems are studied: (1) low-rank matrix completion and (2) neural network learning.