Validation of Automatically Generated Patches: An Appetizer
This tackles the labor-intensive manual inspection needed for widespread adoption of automated program repair tools, though it is incremental as it builds on existing anti-pattern research.
The paper addresses the problem of automatically validating patches in automated program repair to filter out overfitted patches that pass tests but do not fix bugs, proposing two directions: a statistical technique achieving at least 57% precision and a semi-automatic method for stress testing.
In the context of test case based automated program repair (APR), the research community call the patches that pass all the test cases but fail to actually fix the bug test case overfitted patches. Currently, overfitted patches has to be manually inspected by the users. Being a labor intensive activity that hinders widespread adoption of APR tools, automatic validation of APR-generated patches has been the topic of research in recent years. In this paper, we point out the limitations of the existing techniques/methodologies that call for further research, and introduce two promising directions toward effective automatic patch validation: (1) motivated by the relative effectiveness of anti-patterns, we propose to use statistical techniques to avoid the uncomputability of applying some of the anti-pattern rules and automate the technique. Our results show that we achieve at least 57% precision. (2) We present a proposal for a semi-automatic technique that helps the programmers in finding properties of the patched methods and stress testing the patches based on those properties so as to filter out overfitted ones as many as possible.