Assessing the Quality of the Steps to Reproduce in Bug Reports
This addresses a specific issue for software developers by reducing manual effort in bug triage, though it is incremental as it builds on existing research in bug report quality.
The paper tackles the problem of low-quality steps to reproduce in bug reports, which increases manual effort in bug resolution, by proposing Euler, an automated approach that assesses and provides feedback on these steps. The results show that Euler correctly identified 98% of existing steps and 58% of missing ones, with 73% accuracy in quality annotations.
A major problem with user-written bug reports, indicated by developers and documented by researchers, is the (lack of high) quality of the reported steps to reproduce the bugs. Low-quality steps to reproduce lead to excessive manual effort spent on bug triage and resolution. This paper proposes Euler, an approach that automatically identifies and assesses the quality of the steps to reproduce in a bug report, providing feedback to the reporters, which they can use to improve the bug report. The feedback provided by Euler was assessed by external evaluators and the results indicate that Euler correctly identified 98% of the existing steps to reproduce and 58% of the missing ones, while 73% of its quality annotations are correct.