Technical Report: The Policy Graph Improvement Algorithm
This is an incremental technical report that makes existing PGI algorithms more accessible for practitioners and students in AI and robotics.
The authors tackled the challenge of optimizing policies in partially observable Markov decision processes (POMDPs) by describing the policy graph improvement (PGI) algorithm, which uses a fixed-size policy graph to enable monotonic improvement with predictable computation time and compact policies.
Optimizing a partially observable Markov decision process (POMDP) policy is challenging. The policy graph improvement (PGI) algorithm for POMDPs represents the policy as a fixed size policy graph and improves the policy monotonically. Due to the fixed policy size, computation time for each improvement iteration is known in advance. Moreover, the method allows for compact understandable policies. This report describes the technical details of the PGI [1] and particle based PGI [2] algorithms for POMDPs in a more accessible way than [1] or [2] allowing practitioners and students to understand and implement the algorithms.