The Point to Which Soft Actor-Critic Converges
This clarifies a theoretical problem for reinforcement learning researchers, but it is incremental as it builds on existing methods.
The paper tackles the relationship between soft actor-critic and soft Q-learning by proving they converge to the same solution under the maximum entropy framework, translating optimization from an arduous to an easier way.
Soft actor-critic is a successful successor over soft Q-learning. While lived under maximum entropy framework, their relationship is still unclear. In this paper, we prove that in the limit they converge to the same solution. This is appealing since it translates the optimization from an arduous to an easier way. The same justification can also be applied to other regularizers such as KL divergence.