DLFuzz: Differential Fuzzing Testing of Deep Learning Systems
This addresses the need for robust testing in safety-critical domains like autonomous driving, offering an incremental improvement over existing methods by eliminating the requirement for cross-referencing oracles.
The paper tackles the problem of testing deep learning systems for reliability by introducing DLFuzz, a differential fuzzing framework that mutates inputs to maximize neuron coverage and prediction differences, resulting in 338.59% more adversarial inputs with 89.82% smaller perturbations and 2.86% higher neuron coverage compared to DeepXplore.
Deep learning (DL) systems are increasingly applied to safety-critical domains such as autonomous driving cars. It is of significant importance to ensure the reliability and robustness of DL systems. Existing testing methodologies always fail to include rare inputs in the testing dataset and exhibit low neuron coverage. In this paper, we propose DLFuzz, the frst differential fuzzing testing framework to guide DL systems exposing incorrect behaviors. DLFuzz keeps minutely mutating the input to maximize the neuron coverage and the prediction difference between the original input and the mutated input, without manual labeling effort or cross-referencing oracles from other DL systems with the same functionality. We present empirical evaluations on two well-known datasets to demonstrate its efficiency. Compared with DeepXplore, the state-of-the-art DL whitebox testing framework, DLFuzz does not require extra efforts to find similar functional DL systems for cross-referencing check, but could generate 338.59% more adversarial inputs with 89.82% smaller perturbations, averagely obtain 2.86% higher neuron coverage, and save 20.11% time consumption.