Reinforcement Learning for a Discrete-Time Linear-Quadratic Control Problem with an Application
This work addresses control and financial optimization problems, but it appears incremental as it extends existing RL methods to specific models with theoretical proofs.
The authors tackled the discrete-time linear-quadratic control problem by using reinforcement learning with an entropy-based exploration cost, proving that the optimal policy is Gaussian. They applied this to a mean-variance asset-liability management problem, demonstrating policy improvement and convergence through a numerical simulation.
We study the discrete-time linear-quadratic (LQ) control model using reinforcement learning (RL). Using entropy to measure the cost of exploration, we prove that the optimal feedback policy for the problem must be Gaussian type. Then, we apply the results of the discrete-time LQ model to solve the discrete-time mean-variance asset-liability management problem and prove our RL algorithm's policy improvement and convergence. Finally, a numerical example sheds light on the theoretical results established using simulations.