Enhancing Deep Learning with Scenario-Based Override Rules: a Case Study
This work addresses safety and reliability issues in DNN-based systems for software development, but it is incremental as it builds on existing scenario-based modeling.
The paper tackles the problem of deep neural networks behaving unexpectedly on unfamiliar inputs by extending DNN-based systems with hand-crafted override rules using scenario-based modeling, demonstrating feasibility through two case studies and proposing an extension for integration.
Deep neural networks (DNNs) have become a crucial instrument in the software development toolkit, due to their ability to efficiently solve complex problems. Nevertheless, DNNs are highly opaque, and can behave in an unexpected manner when they encounter unfamiliar input. One promising approach for addressing this challenge is by extending DNN-based systems with hand-crafted override rules, which override the DNN's output when certain conditions are met. Here, we advocate crafting such override rules using the well-studied scenario-based modeling paradigm, which produces rules that are simple, extensible, and powerful enough to ensure the safety of the DNN, while also rendering the system more translucent. We report on two extensive case studies, which demonstrate the feasibility of the approach; and through them, propose an extension to scenario-based modeling, which facilitates its integration with DNN components. We regard this work as a step towards creating safer and more reliable DNN-based systems and models.