Autonomous sPOMDP Environment Modeling With Partial Model Exploitation
This work provides a more efficient and scalable method for autonomous robots to learn environmental models, reducing the manual effort required for their deployment.
This paper addresses the challenge of autonomously generating state space representations for robotic systems, which are crucial for efficient planning but laborious to design manually. The authors propose a novel state space exploration algorithm, an extension of surprise-based partially-observable Markov Decision Processes (sPOMDP), demonstrating a 31-63% gain in training speed and improved robustness in less deterministic environments.
A state space representation of an environment is a classic and yet powerful tool used by many autonomous robotic systems for efficient and often optimal solution planning. However, designing these representations with high performance is laborious and costly, necessitating an effective and versatile tool for autonomous generation of state spaces for autonomous robots. We present a novel state space exploration algorithm by extending the original surprise-based partially-observable Markov Decision Processes (sPOMDP), and demonstrate its effective long-term exploration planning performance in various environments. Through extensive simulation experiments, we show the proposed model significantly increases efficiency and scalability of the original sPOMDP learning techniques with a range of 31-63% gain in training speed while improving robustness in environments with less deterministic transitions. Our results pave the way for extending sPOMDP solutions to a broader set of environments.