Modular Domain Adaptation

Findings of the Association for Computational Linguistics: ACL 2022, pages 3633 - 3655 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Modular Domain Adaptation Junshen K. Chen Stanford University kevinehc@gmail.com Dallas Card University of Michigan dalc@umich.edu Dan Jurafsky Stanford University jurafsky@stanford.edu Abstract Off-the-shelf models are widely used by com- putational social science researchers to mea- sure properties of text, such as sentiment. However, without access to source data it is difficult to account for domain shift, which represents a threat to validity. Here, we treat domain adaptation as a modular process that involves separate model producers and model consumers, and show how they can indepen- dently cooperate to facilitate more accurate measurements of text. We introduce two lightweight techniques for this scenario, and demonstrate that they reliably increase out-of- domain accuracy on four multi-domain text classification datasets when used with linear and contextual embedding models. We con- clude with recommendations for model pro- ducers and consumers, and release models and replication code to accompany this paper. 1 Introduction Machine learning models for tasks like sentiment analysis and hate speech detection are becoming in- creasingly ubiquitous as off-the-shelf tools, includ- ing as commercial packages or cloud-based APIs. Among other applications, these models are widely used by computational social scientists to obtain standardized measurements of various document properties at scale. However, the problem of do- main shift represents a threat to validity, one which is difficult for practitioners to overcome, especially without access to source data—which may be un- available for reasons of privacy, copyright, or com- mercial interests. In this paper, we propose to treat domain adaptation as a modular process involving both model producers and model consumers, and show how both parties can independently cooperate to produce more reliable measurements. Although this framework applies to any applica- tion involving independent model producers and consumers, we focus here on text-based instru- ments, including both lexicons and supervised text Figure 1: Modular domain adaptation involves both model producers and model consumers, cooperating via a standardized model. classification models. Using multiple datasets and baselines, we show that model consumers can ob- tain more accurate results by using models de- signed to be lightly adapted, and that model pro- ducers can facilitate such adaptation, even without providing access to source data, using what we call anticipatory domain adaptation (see Figure 1). We introduce two techniques under this new paradigm: domain-specific bias (DSBIAS) and domain-specific normalization (DSNORM). These methods enable model consumers to incorporate information from their domain of interest—without additional training or hyperparameter tuning—and provide reliably better out-of-domain accuracy for both linear and contextual embedding classifiers. In summary, this paper makes the following con- tributions: • We present modular domain adaptation as a process that involves both model producers and model consumers (§3.1). • We introduce two simple techniques for antic- ipatory domain adaptation – that is, ways in which model producers can facilitate adapta- tion by model consumers (§3.4). • We quantify the relative out-of-domain per- formance of linear and contextual embedding models …

Content truncated to 500 words