Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis

Findings of the Association for Computational Linguistics: ACL 2022, pages 3599 - 3610 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis Kai Zhang1, Kun Zhang2, Mengdi Zhang3, Hongke Zhao4, Qi Liu1,∗ Wei Wu3, Enhong Chen1 1 School of Data Science, University of Science and Technology of China 2 School of Computer Science and Information Engineering, Hefei University of Technology 3 Meituan; 4 College of Management and Economics, Tianjin University kkzhang0808@mail.ustc.edu.cn; {qiliuql,cheneh}@ustc.edu.cn {zhang1028kun,wuwei19850318,mdz}@gmail.com; hongke@tju.edu.cn Abstract Aspect-based sentiment analysis (ABSA) pre- dicts sentiment polarity towards a specific as- pect in the given sentence. While pre-trained language models such as BERT have achieved great success, incorporating dynamic semantic changes into ABSA remains challenging. To this end, in this paper, we propose to address this problem by Dynamic Re-weighting BERT (DR-BERT), a novel method designed to learn dynamic aspect-oriented semantics for ABSA. Specifically, we first take the Stack-BERT lay- ers as a primary encoder to grasp the overall semantic of the sentence and then fine-tune it by incorporating a lightweight Dynamic Re- weighting Adapter (DRA). Note that the DRA can pay close attention to a small region of the sentences at each step and re-weigh the vitally important words for better aspect-aware senti- ment understanding. Finally, experimental re- sults on three benchmark datasets demonstrate the effectiveness and the rationality of our pro- posed model and provide good interpretable in- sights for future semantic modeling. 1 Introduction Aspect-based sentiment analysis is a branch of sen- timent analysis, which aims to identify sentiment polarity of the specific aspect in a sentence (Jiang et al., 2011). For example, given a sentence “The restaurant has attentive service, but the food is terrible.”, the task aims to predict the sentiment polarities towards “service” and “food”, which should be positive and negative respectively. As a fundamental technology, the ABSA task has broad applications, such as recommender sys- tem (Chin et al., 2018; Zhang et al., 2021b) and question answering (Wang et al., 2019). Therefore, a great amount of research has been attracted from both academia and industry. Among them, deep neural networks (DNN) (Nguyen and Shirai, 2015; ∗ Corresponding author. Tang et al., 2015, 2016; Zheng et al., 2020), at- tention mechanism (Wang et al., 2016; Ma et al., 2017) and graph neural/attention networks (Huang and Carley, 2019; Zhang et al., 2019a; Wang et al., 2020) have significantly improved the performance through deep feature alignment between the aspect representations and context representations. Recently, the large-scaled pre-trained language models, such as Bidirectional Encoder Represen- tations from Transformers (BERT) (Devlin et al., 2019), realize a breakthrough for improving many language tasks, which further attracts considerable attention to enhance the semantic representations. In ABSA, Xu et al. (2019a) designed BERT-PT, which explores a novel post-training approach on the BERT model. Song et al. (2019) further pro- posed a text pair classification model BERT-SPC, which prepares the input sequence by appending the aspects into the contextual sentence. Although great success has been achieved …

Content truncated to 500 words