Uncertainty Maximization in Partially Observable Domains: A Cognitive Perspective
This work addresses efficiency issues for AI agents in complex environments, though it appears incremental as it builds on existing temporal difference methods.
The paper tackles the problem of redundant information in partially observable domains by selectively focusing on causal interactions, resulting in a significant improvement in convergence of temporal difference algorithms.
Faced with an ever-increasing complexity of their domains of application, artificial learning agents are now able to scale up in their ability to process an overwhelming amount of information coming from their interaction with an environment. However, this process of scaling does come with a cost of encoding and processing an increasing amount of redundant information that is not necessarily beneficial to the learning process itself. This work exploits the properties of the learning systems defined over partially observable domains by selectively focusing on the specific type of information that is more likely to express the causal interaction among the transitioning states of the environment. Adaptive masking of the observation space based on the temporal difference displacement criterion enabled a significant improvement in convergence of temporal difference algorithms defined over a partially observable Markov process.