Shuffling Heuristic in Variational Inequalities: Establishing New Convergence Guarantees
This work addresses a theoretical gap for researchers in optimization and machine learning, offering incremental improvements in convergence analysis for variational inequality algorithms.
The paper tackled the lack of theoretical guarantees for the shuffling heuristic in variational inequalities by providing the first convergence estimates, demonstrating faster convergence in experiments compared to independent sampling methods.
Variational inequalities have gained significant attention in machine learning and optimization research. While stochastic methods for solving these problems typically assume independent data sampling, we investigate an alternative approach -- the shuffling heuristic. This strategy involves permuting the dataset before sequential processing, ensuring equal consideration of all data points. Despite its practical utility, theoretical guarantees for shuffling in variational inequalities remain unexplored. We address this gap by providing the first theoretical convergence estimates for shuffling methods in this context. Our analysis establishes rigorous bounds and convergence rates, extending the theoretical framework for this important class of algorithms. We validate our findings through extensive experiments on diverse benchmark variational inequality problems, demonstrating faster convergence of shuffling methods compared to independent sampling approaches.