Synchronization of mean-field models on the circle
This work addresses synchronization in mean-field models, which is incremental as it generalizes the Kuramoto model and resolves a specific conjecture for transformer-like dynamics.
The paper tackles the problem of global synchronization in mean-field models on the unit circle, proposing a general criterion based on the L1-norm of the third derivative of the interaction function, and applies it to resolve a conjecture for self-attention dynamics by extending the synchronization bound from β in [0,1] to β ≥ -0.16 and showing no synchronization for β < -2/3.
This paper considers a mean-field model of $n$ interacting particles whose state space is the unit circle, a generalization of the classical Kuramoto model. Global synchronization is said to occur if after starting from almost any initial state, all particles coalesce to a common point on the circle. We propose a general synchronization criterion in terms of $L_1$-norm of the third derivative of the particle interaction function. As an application we resolve a conjecture for the so-called self-attention dynamics (stylized model of transformers), by showing synchronization for all $β\ge -0.16$, which significantly extends the previous bound of $0\le β\le 1$ from Criscitiello, Rebjock, McRae, and Boumal (2024). We also show that global synchronization does not occur when $β< -2/3$.