Data-Driven Estimation of Conditional Expectations, Application to Optimal Stopping and Reinforcement Learning
This provides a practical solution for scenarios where analytical knowledge of conditional densities is unavailable, though it appears incremental as it builds on existing data-driven estimation techniques.
The paper tackles the problem of estimating conditional expectations directly from training data when the underlying conditional density is unknown, and extends this data-driven method to solve stochastic optimization problems like optimal stopping and reinforcement learning.
When the underlying conditional density is known, conditional expectations can be computed analytically or numerically. When, however, such knowledge is not available and instead we are given a collection of training data, the goal of this work is to propose simple and purely data-driven means for estimating directly the desired conditional expectation. Because conditional expectations appear in the description of a number of stochastic optimization problems with the corresponding optimal solution satisfying a system of nonlinear equations, we extend our data-driven method to cover such cases as well. We test our methodology by applying it to Optimal Stopping and Optimal Action Policy in Reinforcement Learning.