A Micro-Objective Perspective of Reinforcement Learning
This work addresses the problem of handling performance variability and incorporating domain knowledge in RL for researchers and practitioners, though it appears incremental as it builds on existing RL frameworks.
The paper tackles the limitation of standard reinforcement learning (RL) focusing only on expected cumulative reward by introducing micro-objective RL, which incorporates performance distribution and prior knowledge through temporal abstraction, resulting in a more general formalism applicable to various RL tasks.
The standard reinforcement learning (RL) formulation considers the expectation of the (discounted) cumulative reward. This is limiting in applications where we are concerned with not only the expected performance, but also the distribution of the performance. In this paper, we introduce micro-objective reinforcement learning --- an alternative RL formalism that overcomes this issue. In this new formulation, a RL task is specified by a set of micro-objectives, which are constructs that specify the desirability or undesirability of events. In addition, micro-objectives allow prior knowledge in the form of temporal abstraction to be incorporated into the global RL objective. The generality of this formalism, and its relations to single/multi-objective RL, and hierarchical RL are discussed.