Less is more -- the Dispatcher/ Executor principle for multi-task Reinforcement Learning
This work addresses data efficiency and generalization issues in multi-task reinforcement learning, particularly for scenarios with limited data, but it is presented as a position paper with incremental design principles.
The paper tackles the challenge of improving generalization and data efficiency in multi-task reinforcement learning by introducing a dispatcher/executor principle, which partitions the controller into two entities connected by a regularized communication channel, though no concrete numerical results are provided.
Humans instinctively know how to neglect details when it comes to solve complex decision making problems in environments with unforeseeable variations. This abstraction process seems to be a vital property for most biological systems and helps to 'abstract away' unnecessary details and boost generalisation. In this work we introduce the dispatcher/ executor principle for the design of multi-task Reinforcement Learning controllers. It suggests to partition the controller in two entities, one that understands the task (the dispatcher) and one that computes the controls for the specific device (the executor) - and to connect these two by a strongly regularizing communication channel. The core rationale behind this position paper is that changes in structure and design principles can improve generalisation properties and drastically enforce data-efficiency. It is in some sense a 'yes, and ...' response to the current trend of using large neural networks trained on vast amounts of data and bet on emerging generalisation properties. While we agree on the power of scaling - in the sense of Sutton's 'bitter lesson' - we will give some evidence, that considering structure and adding design principles can be a valuable and critical component in particular when data is not abundant and infinite, but is a precious resource.