Learning to Bid Long-Term: Multi-Agent Reinforcement Learning with Long-Term and Sparse Reward in Repeated Auction Games
This work addresses auction game optimization for agents, but it is incremental as it builds on existing multi-agent reinforcement learning methods with specific reward modifications.
The paper tackled the problem of balancing short-term and sparse long-term rewards in multi-agent reinforcement learning for repeated auction games, demonstrating that their algorithm outperforms benchmarks in direct competition and can be guided to maximize both individual payoff and social welfare.
We propose a multi-agent distributed reinforcement learning algorithm that balances between potentially conflicting short-term reward and sparse, delayed long-term reward, and learns with partial information in a dynamic environment. We compare different long-term rewards to incentivize the algorithm to maximize individual payoff and overall social welfare. We test the algorithm in two simulated auction games, and demonstrate that 1) our algorithm outperforms two benchmark algorithms in a direct competition, with cost to social welfare, and 2) our algorithm's aggressive competitive behavior can be guided with the long-term reward signal to maximize both individual payoff and overall social welfare.