Automatic Error Classification and Root Cause Determination while Replaying Recorded Workload Data at SAP HANA
This addresses the issue of unreliable and time-consuming analysis for software quality assurance teams at SAP HANA, but it is incremental as it applies existing ML methods to a specific domain problem.
The paper tackles the problem of false positive alerts in replaying recorded workloads for database testing at SAP HANA, resulting in a machine learning approach that reduces manual effort and improves quality assurance.
Capturing customer workloads of database systems to replay these workloads during internal testing can be beneficial for software quality assurance. However, we experienced that such replays can produce a large amount of false positive alerts that make the results unreliable or time consuming to analyze. Therefore, we design a machine learning based approach that attributes root causes to the alerts. This provides several benefits for quality assurance and allows for example to classify whether an alert is true positive or false positive. Our approach considerably reduces manual effort and improves the overall quality assurance for the database system SAP HANA. We discuss the problem, the design and result of our approach, and we present practical limitations that may require further research.