Risk-Sensitive Compact Decision Trees for Autonomous Execution in Presence of Simulated Market Response
This work addresses the challenge of costly order execution for traders in financial markets, presenting an incremental improvement by applying risk-sensitive Q-learning to a known market simulation setup.
The paper tackles the problem of minimizing execution costs for high-volume orders in limit order book markets by developing a risk-sensitive reinforcement learning approach that learns trading signals from market microstructure and derives explainable decision-tree-based execution policies, achieving reduced costs with constraints on variance.
We demonstrate an application of risk-sensitive reinforcement learning to optimizing execution in limit order book markets. We represent taking order execution decisions based on limit order book knowledge by a Markov Decision Process; and train a trading agent in a market simulator, which emulates multi-agent interaction by synthesizing market response to our agent's execution decisions from historical data. Due to market impact, executing high volume orders can incur significant cost. We learn trading signals from market microstructure in presence of simulated market response and derive explainable decision-tree-based execution policies using risk-sensitive Q-learning to minimize execution cost subject to constraints on cost variance.