Algorithmic pricing with independent learners and relative experience replay
This addresses algorithmic collusion concerns in economics and AI, offering insights for regulatory and competitive dynamics, though it is incremental in method adaptation.
The study investigated algorithmic collusion in pricing games by incorporating relative performance considerations into reinforcement learning agents, finding that agents averse to underperformance converged to competitive equilibrium while tolerant ones led to supra-competitive prices, with relative experience replay mitigating overfitting in Q-learning.
In an infinitely repeated general-sum pricing game, independent reinforcement learners may exhibit collusive behavior without any communication, raising concerns about algorithmic collusion. To better understand the learning dynamics, we incorporate agents' relative performance (RP) among competitors using experience replay (ER) techniques. Experimental results indicate that RP considerations play a critical role in long-run outcomes. Agents that are averse to underperformance converge to the Bertrand-Nash equilibrium, while those more tolerant of underperformance tend to charge supra-competitive prices. This finding also helps mitigate the overfitting issue in independent Q-learning. Additionally, the impact of relative ER varies with the number of agents and the choice of algorithms.