Shallow Domain Adaptive Embeddings for Sentiment Analysis

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5549–5558, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5549 Shallow Domain Adaptive Embeddings for Sentiment Analysis Prathusha K Sarma, Yingyu Liang and William A Sethares University of Wisconsin-Madison {kameswarasar,sethares}@wisc.edu, yliang@cs.wisc.edu Abstract This paper proposes a way to improve the per- formance of existing algorithms for text clas- sification in domains with strong language se- mantics. We propose a domain adaptation layer learns weights to combine a generic and a domain specific (DS) word embedding into a domain adapted (DA) embedding. The DA word embeddings are then used as inputs to a generic encoder + classifier framework to per- form a downstream task such as classification. This adaptation layer is particularly suited to datasets that are modest in size, and which are, therefore, not ideal candidates for (re)training a deep neural network architecture. Results on binary and multi-class classification tasks using popular encoder architectures, includ- ing current state-of-the-art methods (with and without the shallow adaptation layer) show the effectiveness of the proposed approach. 1 Introduction Domain Adaptation (DA) algorithms are becom- ing increasingly relevant in addressing issues re- lated to i) lack of availability of training data in domains of interest and in ii) exploiting domain idiosyncrasies to improve performance of out-of- domain algorithms. While some state of the art DA algorithms focus on improving on a down- stream task, such as classification in bilingual or crosslingual framework (Hangya et al., 2018) (Liu et al., 2018), others (Zeng et al., 2018), (Shang et al., 2018) focus on tackling DA at the word level for various downstream applications such as clas- sification, tagging etc. Particularly, work by (An et al., 2018) and (K Sarma et al., 2018) address the issue of context-based domain adaptation within a single language. The authors of both works argue that word context causes significant changes, espe- cially in document level sentiment associations. The idea is illustrated by providing examples of words such as ‘kill,’ that has a more positive sen- timent in a document describing video games than when used in a news article. Similarly, words used in social media and medical domains tend to be atypical and idiosyncratic as compared to generic language use. DA algorithms and related transfer learning al- gorithms are often successfully used to perform sentiment analysis, where given a document, the sentiment label for the document is to be deter- mined. Since adaptation can be viewed as a shift in the position of words within the embeddings spaces across the domains of interest, shifts in the spatial position of the words can be measured to verify that the algorithm is indeed capturing do- main knowledge. Once it is established that a given DA method is capturing domain semantics at the word level, the DA word embeddings can then be used as input to an encoder+classification layer to perform sentiment analysis. This paper applies a generic …

Content truncated to 500 words