Model-based Reinforcement Learning for Predictions and Control for Limit Order Books
This addresses the challenge of minimizing risk and monetary loss in algorithmic trading for financial markets, though it appears incremental as it builds on existing model-based RL approaches.
The paper tackled the problem of building a profitable electronic trading agent for stock markets using model-based reinforcement learning, where the agent learns a trading policy by interacting with a simulated environment model built from historical data, and demonstrated that this policy can be transferred to real markets while maintaining profitability.
We build a profitable electronic trading agent with Reinforcement Learning that places buy and sell orders in the stock market. An environment model is built only with historical observational data, and the RL agent learns the trading policy by interacting with the environment model instead of with the real-market to minimize the risk and potential monetary loss. Trained in unsupervised and self-supervised fashion, our environment model learned a temporal and causal representation of the market in latent space through deep neural networks. We demonstrate that the trading policy trained entirely within the environment model can be transferred back into the real market and maintain its profitability. We believe that this environment model can serve as a robust simulator that predicts market movement as well as trade impact for further studies.