Reward Bound for Behavioral Guarantee of Model-based Planning Agents
This work addresses safety assurance for machine learning agents in robotics, offering a theoretical foundation for trustworthiness, though it appears incremental as it builds on existing model-based planning frameworks.
The paper tackles the problem of providing behavioral guarantees for model-based planning agents, specifically ensuring they reach a goal state within a set time, and shows that a lower bound on the reward at the goal state is necessary for such guarantees, with extensions to multiple goals.
Recent years have seen an emerging interest in the trustworthiness of machine learning-based agents in the wild, especially in robotics, to provide safety assurance for the industry. Obtaining behavioral guarantees for these agents remains an important problem. In this work, we focus on guaranteeing a model-based planning agent reaches a goal state within a specific future time step. We show that there exists a lower bound for the reward at the goal state, such that if the said reward is below that bound, it is impossible to obtain such a guarantee. By extension, we show how to enforce preferences over multiple goals.