Why are Some Bugs Non-Reproducible? An Empirical Investigation using Data Fusion
This addresses the problem of unreliable software systems for developers by providing empirical insights into bug non-reproducibility, though it is incremental as it builds on limited prior research.
The paper investigates factors causing non-reproducible software bugs through an empirical study of 576 bug reports from Firefox and Eclipse, identifying 11 key factors, and a user study with 13 developers, offering actionable insights like a false-positive detector to improve reproducibility.
Software developers attempt to reproduce software bugs to understand their erroneous behaviours and to fix them. Unfortunately, they often fail to reproduce (or fix) them, which leads to faulty, unreliable software systems. However, to date, only a little research has been done to better understand what makes the software bugs non-reproducible. In this paper, we conduct a multimodal study to better understand the non-reproducibility of software bugs. First, we perform an empirical study using 576 non-reproducible bug reports from two popular software systems (Firefox, Eclipse) and identify 11 key factors that might lead a reported bug to non-reproducibility. Second, we conduct a user study involving 13 professional developers where we investigate how the developers cope with non-reproducible bugs. We found that they either close these bugs or solicit for further information, which involves long deliberations and counter-productive manual searches. Third, we offer several actionable insights on how to avoid non-reproducibility (e.g., false-positive bug report detector) and improve reproducibility of the reported bugs (e.g., sandbox for bug reproduction) by combining our analyses from multiple studies (e.g., empirical study, developer study).