Scenario-Assisted Deep Reinforcement Learning
This work addresses the opacity and compliance issues in deep reinforcement learning for human engineers, though it appears incremental as it builds on existing methods with a focus on transparency.
The paper tackles the problem of ensuring deep reinforcement learning agents adhere to human-defined constraints by proposing a technique that integrates expert knowledge into the reward calculation, making agents more compliant with requirements. They evaluated it in an internet congestion control case-study, obtaining promising results.
Deep reinforcement learning has proven remarkably useful in training agents from unstructured data. However, the opacity of the produced agents makes it difficult to ensure that they adhere to various requirements posed by human engineers. In this work-in-progress report, we propose a technique for enhancing the reinforcement learning training process (specifically, its reward calculation), in a way that allows human engineers to directly contribute their expert knowledge, making the agent under training more likely to comply with various relevant constraints. Moreover, our proposed approach allows formulating these constraints using advanced model engineering techniques, such as scenario-based modeling. This mix of black-box learning-based tools with classical modeling approaches could produce systems that are effective and efficient, but are also more transparent and maintainable. We evaluated our technique using a case-study from the domain of internet congestion control, obtaining promising results.