KESA: A Knowledge Enhanced Approach To Sentiment Analysis

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 766–776 November 20–23, 2022. ©2022 Association for Computational Linguistics 766 KESA: A Knowledge Enhanced Approach To Sentiment Analysis Qinghua Zhao, Shuai Ma, Shuo Ren SKLSDE Lab, Beihang University, Beijing, China {zhaoqh, mashuai, shuoren}@buaa.edu.cn Abstract Though some recent works focus on inject- ing sentiment knowledge into pre-trained lan- guage models, they usually design mask and reconstruction tasks in the post-training phase. This paper aims to integrate sentiment knowl- edge in the fine-tuning stage. To achieve this goal, we propose two sentiment-aware aux- iliary tasks named sentiment word selection and conditional sentiment prediction and, cor- respondingly, integrate them into the objec- tive of the downstream task. The first task learns to select the correct sentiment words from the given options. The second task pre- dicts the overall sentiment polarity, with the sentiment polarity of the word given as prior knowledge. In addition, two label combina- tion methods are investigated to unify multiple types of labels in each auxiliary task. Experi- mental results demonstrate that our approach consistently outperforms baselines (achieving a new state-of-the-art) and is complementary to existing sentiment-enhanced post-trained models. The codes are released at https: //github.com/lshowway/KESA. 1 Introduction Sentence-level sentiment analysis aims to classify the overall sentiment of a sentence, which has re- ceived considerable attention in natural language processing (Liu, 2012; Zhang et al., 2018, 2022b). Recently, pre-trained language models (PLMs) have achieved state-of-the-art (SOTA) performance on many natural language processing (NLP) tasks, including sentiment analysis. However, it is still challenging to integrate external knowledge into PLMs (Lei et al., 2018; Xu et al., 2019a; Liu et al., 2020b; Wei et al., 2021; Yang et al., 2021; Cui et al., 2021; Zhang et al., 2022a). Recently, sentiment dictionary, a commonly used sentiment knowledge, has been injected into PLMs (Wu et al., 2022). A common practice is to post-train (Xu et al., 2019b), i.e., continue pre- training, self-designed tasks on domain-specific corpora. These tasks include sentiment word pre- diction task, word sentiment prediction task, or aspect-sentiment pairs prediction (Xu et al., 2019a; Tian et al., 2020; Ke et al., 2020; Gururangan et al., 2020; Gu et al., 2020; Tian et al., 2021; Li et al., 2021), just to name a few. Specifically, they are usually designed according to the paradigm of the mask language model (MLM), where sentiment words are first masked and then recovered (includ- ing their polarities) in the output layer. Though ef- fective, we argue that these methods can be further boosted by directly injecting sentiment knowledge, e.g., sentiment polarity, into the output layer when fine-tuning the downstream tasks. In this paper, we aim to inject sentiment knowl- edge into the fine-tuning phase directly, making it complementary to existing methods. For this aim, we propose two novel auxiliary tasks. The first task is sentiment word selection (SWS), aiming to select the sentiment words that belong to the …

Content truncated to 500 words