Tsallis Entropy Regularization for Linearly Solvable MDP and Linear Quadratic Regulator
This work addresses control theory challenges for robotics or AI systems, but it is incremental as it extends existing entropy regularization methods with a parameterized alternative.
The paper tackles the problem of balancing exploration and sparsity in optimal control by applying Tsallis entropy regularization to linearly solvable MDPs and linear quadratic regulators, deriving solutions that demonstrate this trade-off.
Shannon entropy regularization is widely adopted in optimal control due to its ability to promote exploration and enhance robustness, e.g., maximum entropy reinforcement learning known as Soft Actor-Critic. In this paper, Tsallis entropy, which is a one-parameter extension of Shannon entropy, is used for the regularization of linearly solvable MDP and linear quadratic regulators. We derive the solution for these problems and demonstrate its usefulness in balancing between exploration and sparsity of the obtained control law.