Learning to Operate in Open Worlds by Adapting Planning Models
This addresses the issue of planning agents operating in open worlds for AI systems, but it is incremental as it builds on existing novelty detection and model adaptation methods.
The paper tackles the problem of planning agents failing in novel situations where their domain model becomes inaccurate, by introducing an approach that detects novelties and adapts models and actions. The results show it can handle a class of novelties quickly and interpretably on the CartPole benchmark.
Planning agents are ill-equipped to act in novel situations in which their domain model no longer accurately represents the world. We introduce an approach for such agents operating in open worlds that detects the presence of novelties and effectively adapts their domain models and consequent action selection. It uses observations of action execution and measures their divergence from what is expected, according to the environment model, to infer existence of a novelty. Then, it revises the model through a heuristics-guided search over model changes. We report empirical evaluations on the CartPole problem, a standard Reinforcement Learning (RL) benchmark. The results show that our approach can deal with a class of novelties very quickly and in an interpretable fashion.