Complexity Issues and Randomization Strategies in Frank-Wolfe Algorithms for Machine Learning
This work offers incremental improvements for researchers and practitioners using Frank-Wolfe algorithms in optimization and machine learning applications.
The paper investigates the effectiveness of randomization strategies to address complexity issues in Frank-Wolfe algorithms for large-scale machine learning, providing experimental analysis and guidelines based on the results.
Frank-Wolfe algorithms for convex minimization have recently gained considerable attention from the Optimization and Machine Learning communities, as their properties make them a suitable choice in a variety of applications. However, as each iteration requires to optimize a linear model, a clever implementation is crucial to make such algorithms viable on large-scale datasets. For this purpose, approximation strategies based on a random sampling have been proposed by several researchers. In this work, we perform an experimental study on the effectiveness of these techniques, analyze possible alternatives and provide some guidelines based on our results.