Multi-Objective reward generalization: Improving performance of Deep Reinforcement Learning for applications in single-asset trading
This addresses the challenge of reward sparsity and instability in financial trading algorithms, though it appears incremental as an extension of existing multi-objective methods to this domain.
The paper tackles the problem of improving deep reinforcement learning for single-asset trading by proposing a multi-objective algorithm that generalizes reward functions and discount factors, showing increased predictive stability and better performance in sparse reward scenarios compared to single-objective strategies.
We investigate the potential of Multi-Objective, Deep Reinforcement Learning for stock and cryptocurrency single-asset trading: in particular, we consider a Multi-Objective algorithm which generalizes the reward functions and discount factor (i.e., these components are not specified a priori, but incorporated in the learning process). Firstly, using several important assets (cryptocurrency pairs BTCUSD, ETHUSDT, XRPUSDT, and stock indexes AAPL, SPY, NIFTY50), we verify the reward generalization property of the proposed Multi-Objective algorithm, and provide preliminary statistical evidence showing increased predictive stability over the corresponding Single-Objective strategy. Secondly, we show that the Multi-Objective algorithm has a clear edge over the corresponding Single-Objective strategy when the reward mechanism is sparse (i.e., when non-null feedback is infrequent over time). Finally, we discuss the generalization properties with respect to the discount factor. The entirety of our code is provided in open source format.