Device Modeling Bias in ReRAM-based Neural Network Simulations
It addresses the problem of inaccurate hardware simulations for researchers in neuromorphic computing, but is incremental as it builds on existing modeling techniques.
This work investigates how data-driven jump table models for ReRAM devices introduce modeling bias in neural network simulations, showing that binning-based models can unpredictably over- or under-predict network accuracy, particularly with small datasets, as demonstrated on an MNIST-trained multi-layer perceptron.
Data-driven modeling approaches such as jump tables are promising techniques to model populations of resistive random-access memory (ReRAM) or other emerging memory devices for hardware neural network simulations. As these tables rely on data interpolation, this work explores the open questions about their fidelity in relation to the stochastic device behavior they model. We study how various jump table device models impact the attained network performance estimates, a concept we define as modeling bias. Two methods of jump table device modeling, binning and Optuna-optimized binning, are explored using synthetic data with known distributions for benchmarking purposes, as well as experimental data obtained from TiOx ReRAM devices. Results on a multi-layer perceptron trained on MNIST show that device models based on binning can behave unpredictably particularly at low number of points in the device dataset, sometimes over-promising, sometimes under-promising target network accuracy. This paper also proposes device level metrics that indicate similar trends with the modeling bias metric at the network level. The proposed approach opens the possibility for future investigations into statistical device models with better performance, as well as experimentally verified modeling bias in different in-memory computing and neural network architectures.