Combining STPA and BDD for Safety Analysis and Verification in Agile Development: A Controlled Experiment
This addresses safety-critical software development in agile domains, but the results are incremental as they show limited impact beyond communication.
The paper tackled the lack of safety analysis and verification methods in agile development by combining System-Theoretic Process Analysis (STPA) for safety analysis with Behavior Driven Development (BDD) for safety verification, finding that BDD improved communication effectiveness compared to standard User Acceptance Testing (UAT), with no significant differences in productivity, test thoroughness, or fault detection effectiveness in a controlled experiment with 44 students.
Context: Agile development is in widespread use, even in safety-critical domains. Motivation: However, there is a lack of an ap- propriate safety analysis and verification method in agile development. Objective: In this paper, we investigate the use of Behavior Driven De- velopment (BDD) instead of standard User Acceptance Testing (UAT) for safety verification with System-Theoretic Process Analysis (STPA) for safety analysis in agile development. Method: We evaluate the effect of this combination in a controlled experiment with 44 students in terms of productivity, test thoroughness, fault detection effectiveness and com- munication effectiveness. Results: The results show that BDD is more effective for safety verification regarding the impact on communication effectiveness than standard UAT, whereas productivity, test thorough- ness and fault detection effectiveness show no statistically significant difference in our controlled experiment. Conclusion: The combination of BDD and STPA seems promising with an enhancement on communica- tion, but its impact needs more research.