Learnable Conjunction Enhanced Model for Chinese Sentiment Analysis

Learnable Conjunction Enhanced Model for Chinese Sentiment Analysis Bingfei Zhao, Hongying Zan ∗, Jiajia Wang, Yingjie Han Zhengzhou University zbf472670570@163.com, iehyzan@zzu.edu.cn, wjj4work@163.com ieyjhan@zzu.edu.cn Abstract Sentiment analysis is a crucial text classification task that aims to extract, process, and analyze opinions, sentiments, and subjectivity within texts. In current research on Chinese text, sentence and aspect-based sentiment analysis is mainly tackled through well-designed models. However, despite the importance of word order and function words as essential means of semantic ex- pression in Chinese, they are often underutilized. This paper presents a new Chinese sentiment analysis method that utilizes a Learnable Conjunctions Enhanced Model (LCEM). The LCEM adjusts the general structure of the pre-trained language model and incorporates conjunctions location information into the model’s fine-tuning process. Additionally, we discuss a variant structure of residual connections to construct a residual structure that can learn critical informa- tion in the text and optimize it during training. We perform experiments on the public datasets and demonstrate that our approach enhances performance on both sentence and aspect-based sentiment analysis datasets compared to the baseline pre-trained language models. These results confirm the effectiveness of our proposed method. 1 Introduction Sentiment analysis is a crucial area of research within the field of natural language processing. Before the advent of Transformer (Vaswani et al., 2017), Recurrent Neural Networks (RNNs) were the primary method used to model sequences in language modeling tasks (Tang et al., 2016a; Li et al., 2018; Li et al., 2019; Majumder et al., 2022). RNN, along with its variants LSTM (Long-Short Term Memory) and GRU (Gated Recurrent Unit), are powerful models for processing sequences of varying lengths and addressing long-term dependencies. However, the sequential nature of RNNs makes parallelization difficult. Transformer introduces the attention mechanism to encode the context information, which can well capture the internal correlation and ease the problem of long-term dependencies. This allows for greater parallelization and improved performance on certain tasks. Nevertheless, since self-attention discards sequential operations when processing sequences, the po- sition information in the sequence cannot be fully utilized. In languages such as Chinese, word order plays a crucial role in conveying grammatical meaning 0, making it important to consider the sequen- tial nature of the language when developing natural language processing models. Word order refers to the sequence of words in a phrase or sentence, while Chinese word order is relatively fixed, and the change of word order can make the phrase or sentence express different meanings. ”Speak well/说好 话”, ”easy to speak with/好说话”, and ”easier said/话好说” are three Chinese phrases that demonstrate the importance of word order in conveying meaning. Although these phrases share similar characters, their meanings differ greatly depending on how those characters are arranged. ”Speak well/说好话” means to speak positively or say good things about someone or something, while ”easy to speak with/好 说话” describes someone who is easy to communicate with. Lastly, ”easier said/话好说” implies that something may sound simple or easy to do but can be more difficult in practice. It’s essential to consider …

Content truncated to 500 words