Nested Reasoning About Autonomous Agents Using Probabilistic Programs
This addresses the challenge of nested reasoning in autonomous systems, but it is incremental as it builds on existing probabilistic programming and planning methods.
The paper tackles the problem of autonomous agents reasoning about other agents' plans (theory of mind) by developing a planning-as-inference framework using probabilistic programs, applied to pursuit-evasion games, and shows how computation allocation affects estimator variance.
As autonomous agents become more ubiquitous, they will eventually have to reason about the plans of other agents, which is known as theory of mind reasoning. We develop a planning-as-inference framework in which agents perform nested simulation to reason about the behavior of other agents in an online manner. As a concrete application of this framework, we use probabilistic programs to model a high-uncertainty variant of pursuit-evasion games in which an agent must make inferences about the other agents' plans to craft counter-plans. Our probabilistic programs incorporate a variety of complex primitives such as field-of-view calculations and path planners, which enable us to model quasi-realistic scenarios in a computationally tractable manner. We perform extensive experimental evaluations which establish a variety of rational behaviors and quantify how allocating computation across levels of nesting affects the variance of our estimators.