Entropy-Regularized Partially Observed Markov Decision Processes
This work addresses a specific problem in control theory and machine learning for researchers and practitioners dealing with uncertainty in POMDPs, offering incremental improvements by extending existing methods with entropy regularization.
The paper tackles the problem of solving entropy-regularized partially observed Markov decision processes (POMDPs) by showing that standard POMDP techniques provide bounded-error solutions, with exact solutions achievable when regularization involves joint entropy of trajectories, resulting in a tractable formulation for active state estimation.
We investigate partially observed Markov decision processes (POMDPs) with cost functions regularized by entropy terms describing state, observation, and control uncertainty. Standard POMDP techniques are shown to offer bounded-error solutions to these entropy-regularized POMDPs, with exact solutions possible when the regularization involves the joint entropy of the state, observation, and control trajectories. Our joint-entropy result is particularly surprising since it constitutes a novel, tractable formulation of active state estimation.