ReRAM-aware Model Finetuning addressing I-V Non-linearity and Retention Errors
For practitioners deploying large-scale DNNs on ReRAM-based IMC hardware, this work offers a computationally efficient finetuning approach to mitigate non-idealities without training from scratch.
The paper proposes a finetuning-based hardware-aware training algorithm for deploying DNNs on ReRAM crossbar arrays, addressing I-V non-linearity and retention errors. It achieves less than 2% accuracy degradation on ImageNet for MobileNetV3 and only 1 point F-1 score drop on SQuAD v2.
Traditional CPU, GPU, and NPU architectures are increasingly limited by the von Neumann bottleneck. While In-Memory Computing (IMC) using ReRAM crossbar arrays offers a high-density, energy-efficient alternative, its practical deployment is constrained through their non-idealities. Existing hardware-aware training frameworks often require training from scratch, which is computationally prohibitive for modern large-scale models. In this work, we propose a finetuning-based hardware-aware training algorithm that enables robust DNN deployment on ReRAM with minimal training overhead. Our approach mitigates I-V non-linearity by applying a range-shrunk sinh transformation and incorporates retention errors directly into a regularization loss during the finetuning process. We evaluate our framework across models and tasks such as image classification and question-answering (QA). Experimental results demonstrate that our method achieves similar accuracy on large-scale models like ResNet18 and DeiT-Tiny as the base model. In-case of ImageNet for MobileNetV3 families the technique has only less than 2% accuracy degradation. Further, applying the technique on the SQuAD v2 dataset results in only 1 point degradation of F-1 score.