Game-Theory-Assisted Reinforcement Learning for Border Defense: Early Termination based on Analytical Solutions
This addresses sample inefficiency in reinforcement learning for adversarial border defense scenarios, though it appears incremental as it combines existing game theory and RL techniques.
The paper tackles the problem of inefficient reinforcement learning training in border defense games with limited perceptual range by introducing a hybrid approach that uses game-theoretic solutions for early termination of episodes. This method achieves 10-20% higher rewards, faster convergence, and more efficient search trajectories in both single- and multi-defender settings.
Game theory provides the gold standard for analyzing adversarial engagements, offering strong optimality guarantees. However, these guarantees often become brittle when assumptions such as perfect information are violated. Reinforcement learning (RL), by contrast, is adaptive but can be sample-inefficient in large, complex domains. This paper introduces a hybrid approach that leverages game-theoretic insights to improve RL training efficiency. We study a border defense game with limited perceptual range, where defender performance depends on both search and pursuit strategies, making classical differential game solutions inapplicable. Our method employs the Apollonius Circle (AC) to compute equilibrium in the post-detection phase, enabling early termination of RL episodes without learning pursuit dynamics. This allows RL to concentrate on learning search strategies while guaranteeing optimal continuation after detection. Across single- and multi-defender settings, this early termination method yields 10-20% higher rewards, faster convergence, and more efficient search trajectories. Extensive experiments validate these findings and demonstrate the overall effectiveness of our approach.