A Zero-Positive Learning Approach for Diagnosing Software Performance Regressions
This addresses automating performance regression testing for software developers, representing a novel application of machine programming but with incremental technical components.
The paper tackles automating software performance regression testing by introducing AutoPerf, which uses zero-positive learning, autoencoders, and hardware telemetry. It demonstrates an average 4% profiling overhead and accurately diagnoses more performance bugs than prior state-of-the-art approaches with no false negatives.
The field of machine programming (MP), the automation of the development of software, is making notable research advances. This is, in part, due to the emergence of a wide range of novel techniques in machine learning. In this paper, we apply MP to the automation of software performance regression testing. A performance regression is a software performance degradation caused by a code change. We present AutoPerf - a novel approach to automate regression testing that utilizes three core techniques: (i) zero-positive learning, (ii) autoencoders, and (iii) hardware telemetry. We demonstrate AutoPerf's generality and efficacy against 3 types of performance regressions across 10 real performance bugs in 7 benchmark and open-source programs. On average, AutoPerf exhibits 4% profiling overhead and accurately diagnoses more performance bugs than prior state-of-the-art approaches. Thus far, AutoPerf has produced no false negatives.