Learning to Predict Distributions of Words Across Domains

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 613–623, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Learning to Predict Distributions of Words Across Domains Danushka Bollegala Department of Computer Science University of Liverpool Liverpool, L69 3BX, UK danushka.bollegala@ liverpool.ac.uk David Weir Department of Informatics University of Sussex Falmer, Brighton, BN1 9QJ, UK d.j.weir@ sussex.ac.uk John Carroll Department of Informatics University of Sussex Falmer, Brighton, BN1 9QJ, UK j.a.carroll@ sussex.ac.uk Abstract Although the distributional hypothesis has been applied successfully in many natural language processing tasks, systems using distributional information have been lim- ited to a single domain because the dis- tribution of a word can vary between do- mains as the word’s predominant mean- ing changes. However, if it were pos- sible to predict how the distribution of a word changes from one domain to an- other, the predictions could be used to adapt a system trained in one domain to work in another. We propose an unsuper- vised method to predict the distribution of a word in one domain, given its distribu- tion in another domain. We evaluate our method on two tasks: cross-domain part- of-speech tagging and cross-domain sen- timent classification. In both tasks, our method significantly outperforms compet- itive baselines and returns results that are statistically comparable to current state- of-the-art methods, while requiring no task-specific customisations. 1 Introduction The Distributional Hypothesis, summarised by the memorable line of Firth (1957) – You shall know a word by the company it keeps – has inspired a diverse range of research in natural language pro- cessing. In such work, a word is represented by the distribution of other words that co-occur with it. Distributional representations of words have been successfully used in many language process- ing tasks such as entity set expansion (Pantel et al., 2009), part-of-speech (POS) tagging and chunk- ing (Huang and Yates, 2009), ontology learning (Curran, 2005), computing semantic textual sim- ilarity (Besanc¸on et al., 1999), and lexical infer- ence (Kotlerman et al., 2012). However, the distribution of a word often varies from one domain1 to another. For example, in the domain of portable computer reviews the word lightweight is often associated with positive sen- timent bearing words such as sleek or compact, whereas in the movie review domain the same word is often associated with negative sentiment- bearing words such as superficial or formulaic. Consequently, the distributional representations of the word lightweight will differ considerably be- tween the two domains. In this paper, given the distribution wS of a word w in the source domain S, we propose an unsupervised method for pre- dicting its distribution wT in a different target do- main T . The ability to predict how the distribution of a word varies from one domain to another is vital for numerous adaptation tasks. For example, un- supervised cross-domain sentiment classification (Blitzer et al., 2007; Aue and Gamon, 2005) in- volves using sentiment-labeled user reviews from the source domain, and unlabeled reviews from both …

Content truncated to 500 words