PointEval: On the Impact of Pointer Analysis Frameworks
This work addresses the lack of evaluation for pointer analysis tools, which is crucial for researchers in static analysis to advance the field, though it is incremental as it compares existing frameworks.
The authors tackled the problem of fair comparison between pointer analysis frameworks by evaluating WALA and Doop on DaCapo benchmarks, finding that Doop provides better precision and scalability than WALA, and Soot's IR outperforms Wala's IR in most cases.
Pointer analysis is a foundational analysis leveraged by various static analyses. Therefore, it gathered wide attention in research for decades. Some pointer analysis frameworks are based on succinct declarative specifications. However, these tools are heterogeneous in terms of the underlying intermediate representation (IR), heap abstraction, and programming methodology. This situation complicates a fair comparison of these frameworks and thus hinders further research. Consequently, the literature lacks an evaluation of the strengths and weaknesses of these tools. In this work, we evaluate two major frameworks for pointer analysis, WALA and Doop, on the DaCapo set of benchmarks. We compare the pointer analyses available in Wala and Doop, and conclude that---even though based on a declarative specification---Doop provides a better pointer analysis than Wala in terms of precision and scalability. We also compare the two IRs used in Doop, i.e., Jimple from the Soot framework and IR from the Wala framework. Our evaluation shows that in the majority of the benchmarks Soot's IR gives a more precise and scalable pointer analysis. Finally, we propose a micro-benchmark \emph{PointerBench}, for which we manually validate the points-to statistics to evaluate the results of these tools.