A Note About: Critical Review of BugSwarm for Fault Localization and Program Repair
This work clarifies misconceptions about a dataset important for evaluating software tools, but it is incremental as it responds to prior criticism without introducing new methods or data.
The authors addressed a critical review of the BugSwarm dataset for fault localization and program repair, replicating the review and identifying incorrect claims and assumptions about the dataset.
Datasets play an important role in the advancement of software tools and facilitate their evaluation. BugSwarm is an infrastructure to automatically create a large dataset of real-world reproducible failures and fixes. In this paper, we respond to Durieux and Abreu's critical review of the BugSwarm dataset, referred to in this paper as CriticalReview. We replicate CriticalReview's study and find several incorrect claims and assumptions about the BugSwarm dataset. We discuss these incorrect claims and other contributions listed by CriticalReview. Finally, we discuss general misconceptions about BugSwarm, and our vision for the use of the infrastructure and dataset.