CHIME: Cross-passage Hierarchical Memory Network for Generative Review Question Answering

Proceedings of the 28th International Conference on Computational Linguistics, pages 251–265 Barcelona, Spain (Online), December 8-13, 2020 251 Weighed Domain-Invariant Representation Learning for Cross-domain Sentiment Analysis Minlong Peng, Qi Zhang School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai, China {mlpeng16,qz}@fudan.edu.cn Abstract Cross-domain sentiment analysis is currently a hot topic in both the research and industrial areas. One of the most popular framework for the task is domain-invariant representation learning (DIRL), which aims to learn a distribution-invariant feature representation across domains. However, in this work, we find out that applying DIRL may degrade the cross-domain performance when the label distribution P(Y) changes across domains. To address this problem, we propose a modification to DIRL, obtaining a novel weighted domain-invariant representation learning (WDIRL) framework. We show that it is easy to transfer existing models from the DIRL framework to the WDIRL framework. Empirical studies on extensive cross-domain sentiment analysis tasks verified our statements and showed the effectiveness of our proposed solution. 1 Introduction Sentiment analysis aims to predict sentiment polarity of user-generated data with emotional orientation like movie reviews. The exponentially increase of online reviews makes it an interesting topic in research and industrial areas. However, reviews can span so many different domains and the collection and pre- processing of large amounts of data for new domains is often time-consuming and expensive. Therefore, cross-domain sentiment analysis is currently a hot topic, which aims to transfer knowledge from a label- rich source domain (S) to the label-few target domain (T). In recent years, one of the most popular frameworks for cross-domain sentiment analysis is domain invariant representation learning (DIRL) (Glorot et al., 2011; Fernando et al., 2013; Ganin et al., 2016; Zellinger et al., 2017). Methods of the DIRL framework follow the idea of extracting a domain-invariant feature representation, in which the data distributions of the source and target domains are similar. Based on the resultant representations, they learn the supervised classifier using source rich labeled data. The main difference among these methods is the applied technique to force the feature representations to be domain-invariant. However, in this work, we discover that applying DIRL may degrade the cross-domain performance in the situation that the label distribution P(Y) shifts across domains. Specifically, let X and Y denote the input and label random variable, respectively, and G(X) denote the feature representation of X. We found out that when P(Y) changes across domains while P(X|Y) stays the same, forcing G(X) to be domain-invariant will make G(X) uninformative to Y. This will, in turn, harm the generation of the supervised classifier to the target domain. In addition, for the more general condition that both P(Y) and P(X|Y) shift across domains, we deduced a conflict between the object of making the classification error small and that of making G(X) domain-invariant. We argue that the problem is worthy of studying since the shift of P(Y) exists in many real-world cross-domain sentiment analysis tasks (Glorot et al., 2011). For example, the marginal distribution of …

Content truncated to 500 words