Evaluating Manual Intervention to Address the Challenges of Bug Finding with KLEE
This addresses the problem of improving bug detection efficiency for software security researchers, but it is incremental as it builds on existing symbolic execution methods.
The paper tackles the scalability challenges of symbolic execution in bug finding by evaluating manual intervention with the KLEE engine on a new corpus of over 130 real-world bugs, showing that it can increase code coverage and bug detection in many cases.
Symbolic execution has shown its ability to find security-relevant flaws in software, but faces significant scalability challenges. There is a commonly held belief that manual intervention by an expert can help alleviate these limiting factors. However, there has been little formal investigation of this idea. In this paper, we present our experiences applying the KLEE symbolic execution engine to a new bug corpus, and of using manual intervention to alleviate the issues encountered. Our contributions are (1) Hemiptera, a novel corpus of over 130 bugs in real world software, (2) a comprehensive evaluation of the KLEE symbolic execution engine on Hemiptera with a categorisation of frequently occurring software patterns that are problematic for symbolic execution, and (3) an evaluation of manual mitigations aimed at addressing the underlying issues of symbolic execution. Our experience shows that manual intervention can increase both code coverage and bug detection in many situations. It is not a silver bullet however, and we discuss its limitations and the challenges encountered.