Learning to Contest: Decentralized Robust Fairness in Cooperative MARL via Cross-Attention
For multi-agent reinforcement learning researchers, this work provides a decentralized solution to the open problem of ensuring fairness robustness against free-riders in cooperative MARL, though its effectiveness depends on the degree of contest leverage.
This paper shows that decentralized cooperative MARL can achieve robust fairness against free-riders under graded contention, where a contested resource wastes a fraction of its value. The proposed cross-attention policy (CAN) infers the number of free-riders and responds proportionally, achieving low exploitability (ρ≈1.2-1.5) and high efficiency (0.83-1.0) across contention levels, approaching a centralized oracle without a central allocator.
Fair cooperative multi-agent RL (MARL) teams maximizing egalitarian welfare are exploitable: a single selfish agent free-rides on the surplus fair agents forgo to raise the worst-off. A centralized need-based allocator removes it, but only by taking allocation out of agents' hands; whether decentralized policies can be robust was left open. We show this futility is an artifact of all-or-nothing contention. Under graded contention (a contested resource delivers $1-c$, wasting $c$), we prove that for any $c<1$ a worst-off cooperator that contests a free-rider strictly improves on yielding, so decentralized leverage exists (Prop. 1). Realizing it is a coordination problem under uncertainty: the number of free-riders is unknown and variable, so any fixed rule is dominated. We introduce CAN, a permutation-equivariant cross-attention policy over agents' observed behaviour that infers the number of free-riders and responds proportionally: turn-taking when none, contesting just enough when some. Trained against an adversarial league (PSRO), CAN keeps best-response exploitability low ($ρ\approx1.2$-$1.5$, vs. $ρ=N$ unprotected) across the contention range, wasting almost nothing at $D=0$ (efficiency $\approx1.0$) and retaining most of it at $D\geq1$ (efficiency 0.83-0.96), approaching the centralized oracle on both axes, no central allocator. Fair-MARL learners fail on complementary axes (GGF/FEN yield and are exploitable, SOTO all-contests and wastes), while CAN is both. On two further games we find clear scope, not blanket generality: CAN stays efficient and Pareto-dominates the fair learners, but its robustness holds only in proportion to the contest leverage: strong on a multi-server game, partial when it weakens, absent under winner-take-all (Prop. 1 fails). We also report its fragilities: weak leverage and zero-shot transfer to larger teams degrade it at high contention.