Frequency-Based Hyperparameter Selection in Games
This addresses a critical bottleneck in game theory optimization for machine learning practitioners, offering a principled solution to an underexplored area.
The paper tackles the problem of hyperparameter selection in smooth games, where rotational dynamics make classical tuning strategies ineffective, by proposing Modal LookAhead (MoLA), an extension of LookAhead that adaptively selects hyperparameters using frequency estimation, resulting in accelerated training with minimal computational overhead.
Learning in smooth games fundamentally differs from standard minimization due to rotational dynamics, which invalidate classical hyperparameter tuning strategies. Despite their practical importance, effective methods for tuning in games remain underexplored. A notable example is LookAhead (LA), which achieves strong empirical performance but introduces additional parameters that critically influence performance. We propose a principled approach to hyperparameter selection in games by leveraging frequency estimation of oscillatory dynamics. Specifically, we analyze oscillations both in continuous-time trajectories and through the spectrum of the discrete dynamics in the associated frequency-based space. Building on this analysis, we introduce \emph{Modal LookAhead (MoLA)}, an extension of LA that selects the hyperparameters adaptively to a given problem. We provide convergence guarantees and demonstrate in experiments that MoLA accelerates training in both purely rotational games and mixed regimes, all with minimal computational overhead.