Modular Domain Adaptation
This addresses domain adaptation challenges for computational social science researchers using pre-trained models, though it is incremental in nature.
The paper tackles the problem of domain shift in off-the-shelf models used for text analysis by treating domain adaptation as a modular process between model producers and consumers, and demonstrates that their lightweight techniques reliably increase out-of-domain accuracy on four multi-domain text classification datasets.
Off-the-shelf models are widely used by computational social science researchers to measure properties of text, such as sentiment. However, without access to source data it is difficult to account for domain shift, which represents a threat to validity. Here, we treat domain adaptation as a modular process that involves separate model producers and model consumers, and show how they can independently cooperate to facilitate more accurate measurements of text. We introduce two lightweight techniques for this scenario, and demonstrate that they reliably increase out-of-domain accuracy on four multi-domain text classification datasets when used with linear and contextual embedding models. We conclude with recommendations for model producers and consumers, and release models and replication code to accompany this paper.