Learning Explainable and Better Performing Representations of POMDP Strategies
This work addresses the challenge of explainable and scalable strategy representation in POMDPs for AI and control systems, though it is incremental as it builds on existing automaton and L*-algorithm techniques.
The paper tackles the problem of representing strategies in partially observable Markov decision processes (POMDPs) by learning automaton representations using a modified L*-algorithm, resulting in dramatically smaller and more explainable strategies with potential performance improvements and incomparably greater scalability compared to direct synthesis methods.
Strategies for partially observable Markov decision processes (POMDP) typically require memory. One way to represent this memory is via automata. We present a method to learn an automaton representation of a strategy using a modification of the L*-algorithm. Compared to the tabular representation of a strategy, the resulting automaton is dramatically smaller and thus also more explainable. Moreover, in the learning process, our heuristics may even improve the strategy's performance. In contrast to approaches that synthesize an automaton directly from the POMDP thereby solving it, our approach is incomparably more scalable.