RL as Regressor: A Reinforcement Learning Approach for Function Approximation
This approach addresses the problem of inflexible loss functions in regression for machine learning practitioners, though it appears incremental as it applies existing RL methods to a new context.
The paper tackles the limitation of standard regression techniques by framing regression as a reinforcement learning problem, using a custom reward signal to handle non-differentiable objectives, and demonstrates this through a case study on learning a noisy sine wave, showing successful function approximation with enhanced flexibility.
Standard regression techniques, while powerful, are often constrained by predefined, differentiable loss functions such as mean squared error. These functions may not fully capture the desired behavior of a system, especially when dealing with asymmetric costs or complex, non-differentiable objectives. In this paper, we explore an alternative paradigm: framing regression as a Reinforcement Learning (RL) problem. We demonstrate this by treating a model's prediction as an action and defining a custom reward signal based on the prediction error, and we can leverage powerful RL algorithms to perform function approximation. Through a progressive case study of learning a noisy sine wave, we illustrate the development of an Actor-Critic agent, iteratively enhancing it with Prioritized Experience Replay, increased network capacity, and positional encoding to enable a capable RL agent for this regression task. Our results show that the RL framework not only successfully solves the regression problem but also offers enhanced flexibility in defining objectives and guiding the learning process.