Greedy Frank-Wolfe Algorithm for Exemplar Selection
This addresses the computational bottleneck in exemplar selection for machine learning practitioners, offering a faster and more accurate method, though it appears incremental as it builds on existing Frank-Wolfe and sparse representation techniques.
The paper tackles the problem of selecting a small subset of exemplars from a dataset for learning tasks, proposing the Frank-Wolfe Sparse Representation (FWSR) algorithm that requires only about k iterations with quadratic per-iteration cost, outperforming state-of-the-art methods in speed and accuracy across most tested scenarios.
In this paper, we consider the problem of selecting representatives from a data set for arbitrary supervised/unsupervised learning tasks. We identify a subset $S$ of a data set $A$ such that 1) the size of $S$ is much smaller than $A$ and 2) $S$ efficiently describes the entire data set, in a way formalized via convex optimization. In order to generate $|S| = k$ exemplars, our kernelizable algorithm, Frank-Wolfe Sparse Representation (FWSR), only needs to execute $\approx k$ iterations with a per-iteration cost that is quadratic in the size of $A$. This is in contrast to other state of the art methods which need to execute until convergence with each iteration costing an extra factor of $d$ (dimension of the data). Moreover, we also provide a proof of linear convergence for our method. We support our results with empirical experiments; we test our algorithm against current methods in three different experimental setups on four different data sets. FWSR outperforms other exemplar finding methods both in speed and accuracy in almost all scenarios.