Alignment Adapter to Improve the Performance of Compressed Deep Learning Models
This addresses the problem of deploying efficient models in resource-constrained environments, offering an incremental improvement for domain-specific applications.
The paper tackles the performance gap between compressed and original deep learning models by proposing Alignment Adapter (AlAd), a lightweight adapter that aligns token-level embeddings, and shows it significantly boosts performance on NLP tasks with minimal overhead.
Compressed Deep Learning (DL) models are essential for deployment in resource-constrained environments. But their performance often lags behind their large-scale counterparts. To bridge this gap, we propose Alignment Adapter (AlAd): a lightweight, sliding-window-based adapter. It aligns the token-level embeddings of a compressed model with those of the original large model. AlAd preserves local contextual semantics, enables flexible alignment across differing dimensionalities or architectures, and is entirely agnostic to the underlying compression method. AlAd can be deployed in two ways: as a plug-and-play module over a frozen compressed model, or by jointly fine-tuning AlAd with the compressed model for further performance gains. Through experiments on BERT-family models across three token-level NLP tasks, we demonstrate that AlAd significantly boosts the performance of compressed models with only marginal overhead in size and latency.