RV4Chatbot: Are Chatbots Allowed to Dream of Electric Sheep?
This addresses safety concerns for chatbot users in critical domains, but it is incremental as it applies existing runtime verification methods to chatbots.
The paper tackles the problem of ensuring chatbots adhere to safe behaviors in safety-critical applications by introducing RV4Chatbot, a runtime verification framework that monitors deviations, with implementations for Rasa and Dialogflow tested in a factory automation scenario.
Chatbots have become integral to various application domains, including those with safety-critical considerations. As a result, there is a pressing need for methods that ensure chatbots consistently adhere to expected, safe behaviours. In this paper, we introduce RV4Chatbot, a Runtime Verification framework designed to monitor deviations in chatbot behaviour. We formalise expected behaviours as interaction protocols between the user and the chatbot. We present the RV4Chatbot design and describe two implementations that instantiate it: RV4Rasa, for monitoring chatbots created with the Rasa framework, and RV4Dialogflow, for monitoring Dialogflow chatbots. Additionally, we detail experiments conducted in a factory automation scenario using both RV4Rasa and RV4Dialogflow.