Portfolio Optimization under Recursive Utility via Reinforcement Learning
This work addresses portfolio optimization for investors by introducing a risk-sensitive objective, though it is incremental as it adapts existing RL methods to a specific financial context.
The paper tackled portfolio allocation by applying a risk-sensitive recursive utility objective via reinforcement learning, resulting in improved Sharpe ratio, max drawdown, and cumulative return compared to a discounted baseline on South Korean ETF data.
We study whether a risk-sensitive objective from asset-pricing theory -- recursive utility -- improves reinforcement learning for portfolio allocation. The Bellman equation under recursive utility involves a certainty equivalent (CE) of future value that has no closed form under observed returns; we approximate it by $K$-sample Monte Carlo and train actor-critic (PPO, A2C) on the resulting value target and an approximate advantage estimate (AAE) that generalizes the Bellman residual to multi-step with state-dependent weights. This formulation applies only to critic-based algorithms. On 10 chronological train/test splits of South Korean ETF data, the recursive-utility agent improves on the discounted (naive) baseline in Sharpe ratio, max drawdown, and cumulative return. Derivations, world model and metrics, and full result tables are in the appendices.