HDTest: Differential Fuzz Testing of Brain-Inspired Hyperdimensional Computing
This work addresses the reliability gap for HDC, an emerging energy-efficient alternative to deep neural networks, by providing a systematic testing approach, though it is incremental as it adapts existing fuzz testing to a new domain.
The paper tackles the problem of reliability and robustness in brain-inspired hyperdimensional computing (HDC) models by introducing HDTest, a differential fuzz testing method that automatically generates adversarial inputs to expose incorrect behaviors, resulting in the generation of thousands of adversarial inputs with negligible perturbations, such as around 400 per minute on a commodity computer, and improving model robustness through retraining.
Brain-inspired hyperdimensional computing (HDC) is an emerging computational paradigm that mimics brain cognition and leverages hyperdimensional vectors with fully distributed holographic representation and (pseudo)randomness. Compared to other machine learning (ML) methods such as deep neural networks (DNNs), HDC offers several advantages including high energy efficiency, low latency, and one-shot learning, making it a promising alternative candidate on a wide range of applications. However, the reliability and robustness of HDC models have not been explored yet. In this paper, we design, implement, and evaluate HDTest to test HDC model by automatically exposing unexpected or incorrect behaviors under rare inputs. The core idea of HDTest is based on guided differential fuzz testing. Guided by the distance between query hypervector and reference hypervector in HDC, HDTest continuously mutates original inputs to generate new inputs that can trigger incorrect behaviors of HDC model. Compared to traditional ML testing methods, HDTest does not need to manually label the original input. Using handwritten digit classification as an example, we show that HDTest can generate thousands of adversarial inputs with negligible perturbations that can successfully fool HDC models. On average, HDTest can generate around 400 adversarial inputs within one minute running on a commodity computer. Finally, by using the HDTest-generated inputs to retrain HDC models, we can strengthen the robustness of HDC models. To the best of our knowledge, this paper presents the first effort in systematically testing this emerging brain-inspired computational model.