Vision-based Perimeter Defense via Multiview Pose Estimation
This work addresses the practical challenge of perimeter defense for security applications where defenders must perceive intruders visually, representing an incremental advance over fully observable models.
The paper tackles the problem of perimeter defense in partially observable settings by developing a vision-based system that estimates intruder poses from multiple views, improving state estimation and defense performance in simulated and real-world scenarios.
Previous studies in the perimeter defense game have largely focused on the fully observable setting where the true player states are known to all players. However, this is unrealistic for practical implementation since defenders may have to perceive the intruders and estimate their states. In this work, we study the perimeter defense game in a photo-realistic simulator and the real world, requiring defenders to estimate intruder states from vision. We train a deep machine learning-based system for intruder pose detection with domain randomization that aggregates multiple views to reduce state estimation errors and adapt the defensive strategy to account for this. We newly introduce performance metrics to evaluate the vision-based perimeter defense. Through extensive experiments, we show that our approach improves state estimation, and eventually, perimeter defense performance in both 1-defender-vs-1-intruder games, and 2-defenders-vs-1-intruder games.