Efficient Fine-Tuning of BERT Models on the Edge
This addresses the challenge of on-device adaptability for NLP applications in dynamic environments, though it is incremental as it builds on existing fine-tuning methods.
The paper tackles the problem of fine-tuning large BERT models on resource-constrained edge devices by proposing a memory-efficient training regime called FAR, which reduces fine-tuning time by 30% and memory operation time by 47% on DistilBERT with minimal performance drops of around 1% on benchmarks.
Resource-constrained devices are increasingly the deployment targets of machine learning applications. Static models, however, do not always suffice for dynamic environments. On-device training of models allows for quick adaptability to new scenarios. With the increasing size of deep neural networks, as noted with the likes of BERT and other natural language processing models, comes increased resource requirements, namely memory, computation, energy, and time. Furthermore, training is far more resource intensive than inference. Resource-constrained on-device learning is thus doubly difficult, especially with large BERT-like models. By reducing the memory usage of fine-tuning, pre-trained BERT models can become efficient enough to fine-tune on resource-constrained devices. We propose Freeze And Reconfigure (FAR), a memory-efficient training regime for BERT-like models that reduces the memory usage of activation maps during fine-tuning by avoiding unnecessary parameter updates. FAR reduces fine-tuning time on the DistilBERT model and CoLA dataset by 30%, and time spent on memory operations by 47%. More broadly, reductions in metric performance on the GLUE and SQuAD datasets are around 1% on average.