Adaptive Ensembling: Unsupervised Domain Adaptation for Political Document Analysis

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4718–4730, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4718 Adaptive Ensembling: Unsupervised Domain Adaptation for Political Document Analysis Shrey Desai1, Barea Sinno2, Alex Rosenfeld3, and Junyi Jessy Li3 1Department of Computer Science 2Department of Government, Department of Statistics & Data Science 3Department of Linguistics The University of Texas at Austin shreydesai@utexas.edu, barea.sinno@utexas.edu alexbrosenfeld@gmail.com, jessy@austin.utexas.edu Abstract Insightful findings in political science often re- quire researchers to analyze documents of a certain subject or type, yet these documents are usually contained in large corpora that do not distinguish between pertinent and non- pertinent documents. In contrast, we can find corpora that label relevant documents but have limitations (e.g., from a single source or era), preventing their use for political science re- search. To bridge this gap, we present adaptive ensembling, an unsupervised domain adapta- tion framework, equipped with a novel text classification model and time-aware training to ensure our methods work well with diachronic corpora. Experiments on an expert-annotated dataset show that our framework outperforms strong benchmarks. Further analysis indicates that our methods are more stable, learn bet- ter representations, and extract cleaner corpora for fine-grained analysis. 1 Introduction Recent progress in natural language processing and computational social science have pushed po- litical science research into new frontiers. For example, scholars have studied language use in presidential elections (Acree et al., 2018), legisla- tive text in Congress (de Marchi et al., 2018), and similarities in national constitutions (Elkins and Shaffer, 2019). However, datasets used by polit- ical scientists are mostly homogeneous in terms of subject (e.g., immigration) or document type (e.g., constitutions). Labeled corpora with perti- nent documents usually only stem from a single source; this makes it difficult to generalize con- clusions derived from them to other sources. On the other hand, corpora spanning multiple decades and sources tend to be unlabeled. These cor- pora are largely untouched by political scientists; to illustrate some problems that arise with study- ing such data, Table 1 shows a sample of topics Topic 1 like, day, would, a.m., center Topic 2 two, samour, family, veronica, son Topic 3 would, hospital, also, car, hyundai Topic 4 said, people, one, years, think Topic 5 city, 6-4, last, wine, york Table 1: Randomly sampled topics and top keywords derived from a 50-topic LDA model trained on a sam- ple of COHA documents. Topic modeling results on a political subset of COHA are presented in Table 5. Ad- ditionally, topic model hyperparameters are detailed in Appendix A. generated by Latent Dirichlet Allocation (LDA) (Blei et al., 2003), a popular topic model in so- cial science, trained on 60,000 documents sampled from the Corpus of Historical American English (COHA) (Davies, 2008). The generated topics are extremely vague and not specific to politics. This paper bridges the gap between labeled and unlabeled corpora by framing the problem as one of domain adaptation. We …

Content truncated to 500 words