Communication Efficient Parallel Algorithms for Optimization on Manifolds
This work addresses a critical gap for researchers and practitioners dealing with manifold-based data, though it is incremental as it extends existing parallel methods to a new setting.
The authors tackled the lack of parallel inference algorithms for non-Euclidean data by generalizing them to optimization on manifolds, resulting in a communication-efficient algorithm with theoretical convergence guarantees demonstrated on simulated spherical data and the Netflix dataset.
The last decade has witnessed an explosion in the development of models, theory and computational algorithms for "big data" analysis. In particular, distributed computing has served as a natural and dominating paradigm for statistical inference. However, the existing literature on parallel inference almost exclusively focuses on Euclidean data and parameters. While this assumption is valid for many applications, it is increasingly more common to encounter problems where the data or the parameters lie on a non-Euclidean space, like a manifold for example. Our work aims to fill a critical gap in the literature by generalizing parallel inference algorithms to optimization on manifolds. We show that our proposed algorithm is both communication efficient and carries theoretical convergence guarantees. In addition, we demonstrate the performance of our algorithm to the estimation of Fréchet means on simulated spherical data and the low-rank matrix completion problem over Grassmann manifolds applied to the Netflix prize data set.