Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4933–4941 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4933 Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification Alexander Rietzler, Sebastian Stabinger, Paul Opitz, Stefan Engl DeepOpinion Bozner Platz 1, Innsbruck, Austria {alexander.rietzler, sebastian.stabinger, paul.opitz, stefan.engl}@deepopinion.ai Abstract Aspect-Target Sentiment Classification (ATSC) is a subtask of Aspect-Based Sentiment Analysis (ABSA), which has many applications e.g. in e-commerce, where data and insights from reviews can be leveraged to create value for businesses and customers. Recently, deep transfer-learning methods have been applied successfully to a myriad of Natural Language Processing (NLP) tasks, including ATSC. Building on top of the prominent BERT language model, we approach ATSC using a two-step procedure: self-supervised domain-specific BERT language model finetuning, followed by supervised task-specific finetuning. Our findings on how to best exploit domain-specific language model finetuning enable us to produce new state-of-the-art performance on the SemEval 2014 Task 4 restaurants dataset. In addition, to explore the real-world robustness of our models, we perform cross-domain evaluation. We show that a cross-domain adapted BERT language model performs significantly better than strong baseline models like vanilla BERT-base and XLNet-base. Finally, we conduct a case study to interpret model prediction errors. Keywords: Aspect-Based Sentiment Analysis, Targeted Sentiment Classification, Cross-Domain Adaptation, BERT, XLNet 1. Introduction Sentiment Analysis (SA) is an active field of research in Natural Language Processing and deals with opinions in text. A typical application of classical SA in an industrial setting would be to classify a document like a product re- view into positive, negative or neutral sentiment polarity. In contrast to SA, the more fine-grained task of Aspect- Based Sentiment Analysis (ABSA) (Hu and Liu, 2004; Pontiki et al., 2015) aims to find both the aspect of an en- tity like a restaurant, and the sentiment associated with this aspect. It is important to note that ABSA comes in two vari- ants. We will use the sentence “I love their dumplings” to explain these variants in detail. Both variants are implemented as a two-step procedure. The first variant is comprised of Aspect-Category Detection (ACD) followed by Aspect-Category Sentiment Classifica- tion (ACSC). ACD is a multilabel classification task, where a sentence can be associated with a set of predefined as- pect categories like “food” and “service” in the restaurants domain. In the second step, ACSC, the sentiment polar- ity associated to the aspect-category is classified. For our example-sentence the correct result is the tuple (“food”, “positive”). The second variant consists of Aspect-Target Extraction (ATE) followed by Aspect-Target Sentiment Classification (ATSC). ATE is a sequence labeling task, where terms like “dumplings” are detected. In the second step, ATSC, the sentiment polarity associated with the aspect-target is deter- mined. In our example the correct result is (”dumplings”, ”positive”). In this paper, we focus on ATSC. In recent years, special- ized neural architectures (Tang et al., 2016a; Tang et al., 2016b) have been developed that substantially …

Content truncated to 500 words