Mull it over: mutation testing based on LLVM
This tool addresses the need for efficient, language-independent mutation testing in compiled programming languages, representing an incremental improvement over existing tools.
The authors developed Mull, an open-source mutation testing tool based on LLVM that works with any language compilable to LLVM IR (e.g., C, C++, Rust, Swift), achieving language independence and faster processing by recompiling only modified IR fragments. Their evaluation on real-world projects like RODOS, OpenSSL, and LLVM demonstrated its capabilities, though specific performance numbers were not provided in the abstract.
This paper describes Mull, an open-source tool for mutation testing based on the LLVM framework. Mull works with LLVM IR, a low-level intermediate representation, to perform mutations, and uses LLVM JIT for just-in-time compilation. This design choice enables the following two capabilities of Mull: language independence and fine-grained control over compilation and execution of a tested program and its mutations. Mull can work with code written in any programming language that supports compilation to LLVM IR, such as C, C++, Rust, or Swift. Direct manipulation of LLVM IR allows Mull to do less work to generate mutations: only modified fragments of IR code are recompiled, and this results in faster processing of mutated programs. To our knowledge, no existing mutation testing tool provides these capabilities for compiled programming languages. We describe the algorithm and implementation details of Mull, highlight current limitations of Mull, and present the results of our evaluation of Mull on real-world projects such as RODOS, OpenSSL, LLVM.