Market Making via Reinforcement Learning
This work addresses inventory risk in market making for financial trading, presenting an incremental improvement over existing methods.
The paper tackled the market making problem by developing a reinforcement learning agent with a custom reward function to manage inventory risk, and demonstrated its effectiveness by outperforming benchmark strategies and a recent online learning approach.
Market making is a fundamental trading problem in which an agent provides liquidity by continually offering to buy and sell a security. The problem is challenging due to inventory risk, the risk of accumulating an unfavourable position and ultimately losing money. In this paper, we develop a high-fidelity simulation of limit order book markets, and use it to design a market making agent using temporal-difference reinforcement learning. We use a linear combination of tile codings as a value function approximator, and design a custom reward function that controls inventory risk. We demonstrate the effectiveness of our approach by showing that our agent outperforms both simple benchmark strategies and a recent online learning approach from the literature.