Learning Transition Models with Time-delayed Causal Relations
This addresses data-efficiency and interpretability issues in model-based RL for robotics, representing an incremental improvement.
The paper tackles the problem of discovering implicit and time-delayed causal relations in robot observations to improve model-based reinforcement learning, resulting in significant performance gains over current techniques in simulated and real robotic tasks.
This paper introduces an algorithm for discovering implicit and delayed causal relations between events observed by a robot at arbitrary times, with the objective of improving data-efficiency and interpretability of model-based reinforcement learning (RL) techniques. The proposed algorithm initially predicts observations with the Markov assumption, and incrementally introduces new hidden variables to explain and reduce the stochasticity of the observations. The hidden variables are memory units that keep track of pertinent past events. Such events are systematically identified by their information gains. The learned transition and reward models are then used for planning. Experiments on simulated and real robotic tasks show that this method significantly improves over current RL techniques.