Lyapunov Exponents for Diversity in Differentiable Games
This work addresses the challenge of solution diversity in multi-agent optimization for machine learning, representing an incremental extension of prior methods to non-conservative settings.
The authors tackled the problem of finding diverse solutions in non-conservative, multi-agent gradient systems by proposing Generalized Ridge Rider (GRR), which generalizes an existing algorithm to handle arbitrary bifurcation points, and demonstrated its effectiveness in applications like the iterated prisoner's dilemma and generative adversarial networks.
Ridge Rider (RR) is an algorithm for finding diverse solutions to optimization problems by following eigenvectors of the Hessian ("ridges"). RR is designed for conservative gradient systems (i.e., settings involving a single loss function), where it branches at saddles - easy-to-find bifurcation points. We generalize this idea to non-conservative, multi-agent gradient systems by proposing a method - denoted Generalized Ridge Rider (GRR) - for finding arbitrary bifurcation points. We give theoretical motivation for our method by leveraging machinery from the field of dynamical systems. We construct novel toy problems where we can visualize new phenomena while giving insight into high-dimensional problems of interest. Finally, we empirically evaluate our method by finding diverse solutions in the iterated prisoners' dilemma and relevant machine learning problems including generative adversarial networks.