Nonsmoothness in Machine Learning: specific structure, proximal identification, and applications
This work addresses the problem of optimizing nonsmooth functions in machine learning, which is incremental as it builds on existing knowledge to improve practical applications.
The paper tackles the challenge of nonsmooth optimization in machine learning by analyzing its specific structure and demonstrating how to leverage it for practical benefits like compression, acceleration, or dimension reduction, though no concrete numerical results are provided.
Nonsmoothness is often a curse for optimization; but it is sometimes a blessing, in particular for applications in machine learning. In this paper, we present the specific structure of nonsmooth optimization problems appearing in machine learning and illustrate how to leverage this structure in practice, for compression, acceleration, or dimension reduction. We pay a special attention to the presentation to make it concise and easily accessible, with both simple examples and general results.