Syntactically Aware Cross-Domain Aspect and Opinion Terms Extraction

Proceedings of the 28th International Conference on Computational Linguistics, pages 1772–1777 Barcelona, Spain (Online), December 8-13, 2020 1772 Syntactically Aware Cross-Domain Aspect and Opinion Terms Extraction Oren Pereg, Daniel Korat, Moshe Wasserblat Intel AI Lab, Petah Tikva, Israel {oren.pereg, daniel.korat, moshe.wasserblat}@intel.com Abstract A fundamental task of fine-grained sentiment analysis is aspect and opinion terms extraction. Supervised-learning approaches have shown good results for this task; however, they fail to scale across domains where labeled data is lacking. Non pre-trained unsupervised domain adapta- tion methods that incorporate external linguistic knowledge have proven effective in transferring aspect and opinion knowledge from a labeled source domain to unlabeled target domains; how- ever, pre-trained transformer-based models like BERT and RoBERTa already exhibit substantial syntactic knowledge. In this paper, we propose a method for incorporating external linguistic information into a self-attention mechanism coupled with the BERT model. This enables lever- aging the intrinsic knowledge existing within BERT together with externally introduced syntactic information, to bridge the gap across domains. Finally, we demonstrate enhanced results on three benchmark datasets. 1 Introduction A fundamental task of fine-grained sentiment analysis is aspect and opinion terms extraction. For ex- ample, in the sentence “The chocolate cake was incredible”, the aspect term is chocolate cake and the opinion term is incredible. Most of the work related to aspect and opinion term extraction is formulated as a supervised sequence-tagging task. RNN-based models (Liu et al., 2015) and Transformer-based mod- els showed promising results in single-domain setups where the training and the testing data are from the same domain. However, these approaches typically do not scale across different domains, where only unlabeled data is available for the target domain, since aspect terms from two different domains are usually semantically different hence separated in the embedding space. For example, frequent aspect terms in the restaurant domain, like salad and dessert, have little or no semantic relatedness to frequent aspect terms in the laptop domain, like screen size and battery life. To date, only a handful of approaches for unsupervised domain adaptation of aspect and opinion term extraction have been proposed. It has been shown that syntactic information is important for identifying aspect and opinion terms (Hu and Liu, 2004b; Qiu et al., 2011). A recent line of work, based on non pre-trained models, encodes dependency-based aspect extraction rules (Ding et al., 2017) or automatically-generated dependency relations (Wang and Jialin Pan, 2018; Wang and Pan, 2020), as auxiliary supervision for non pre-trained models. This recent line of work demonstrates effective domain adaptation by incorporating syntactic knowledge into non pre-trained models during their training step. Subsequently, recent studies (Clark et al., 2019; Htut et al., 2019) show that pre-trained transformer-based models such as BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019) already exhibit substantial linguistic knowledge. In this paper we examine whether the incorporation of external syntactic knowledge into pre-trained models, contributes to bridging the gap across domains. For this purpose, we propose an approach for unsupervised domain- adaptation of aspect and opinion …

Content truncated to 500 words