Recommendation Chart of Domains for Cross-Domain Sentiment Analysis: Findings of A 20 Domain Study

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4982–4990 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4982 Recommendation Chart of Domains for Cross-Domain Sentiment Analysis: Findings of A 20 Domain Study Akash Sheoran♦, Diptesh Kanojia†,♣,⋆, Aditya Joshi♥, and Pushpak Bhattacharyya† ♦Samsung Research Institute Bangalore, India †Indian Institute of Technology Bombay, India ♣IITB-Monash Research Academy, India ⋆Monash University, Australia ♥CSIRO Data61, Australia ♦a.sheoran@samsung.com, †{diptesh, pb}@cse.iitb.ac.in, ♥aditya.joshi@data61.csiro.au Abstract Cross-domain sentiment analysis (CDSA) helps to address the problem of data scarcity in scenarios where labelled data for a domain (known as the target domain) is unavailable or insufficient. However, the decision to choose a domain (known as the source domain) to leverage from is, at best, intuitive. In this paper, we investigate text similarity metrics to facilitate source domain selection for CDSA. We report results on 20 domains (all possible pairs) using 11 similarity metrics. Specifically, we compare CDSA performance with these metrics for different domain-pairs to enable the selection of a suitable source domain, given a target domain. These metrics include two novel metrics for evaluating domain adaptability to help source domain selection of labelled data and utilize word and sentence-based embeddings as metrics for unlabelled data. The goal of our experiments is a recommendation chart that gives the K best source domains for CDSA for a given target domain. We show that the best K source domains returned by our similarity metrics have a precision of over 50%, for varying values of K. Keywords: sentiment analysis evaluation, cross-domain sentiment analysis, similarity metrics 1. Introduction Sentiment analysis (SA) deals with automatic detection of opinion orientation in text (Liu and others, 2010). Domain- specificity of sentiment words, and, as a result, sentiment analysis is also a well-known challenge. A popular example being ‘unpredictable’ that is positive for a book review (as in ‘The plot of the book is unpredictable’) but negative for an automobile review (as in ‘The steering of the car is un- predictable’). Therefore, a classifier that has been trained on book reviews may not perform as well for automobile reviews (Pang et al., 2008). However, sufficient datasets may not be available for a domain in which an SA system is to be trained. This has resulted in research in cross-domain sentiment analy- sis (CDSA). CDSA refers to approaches where the training data is from a different domain (referred to as the ‘source domain’) as compared to that of the test data (referred to as the ‘target domain’). Ben-David et al. (2007) show that similarity between the source and target domains can be used as indicators for domain adaptation, in general. In this paper, we validate the idea for CDSA. We use similarity metrics as a basis for source domain selection. We implement an LSTM-based sentiment classifier and evaluate its performance for CDSA for a dataset of reviews from twenty domains. We then compare it with similarity metrics to understand which metrics are useful. The resultant deliverable is a recommendation …

Content truncated to 500 words