MT-Speech at SemEval-2022 Task 10: Incorporating Data Augmentation and Auxiliary Task with Cross-Lingual Pretrained Language Model for Structured Sentiment Analysis

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1329 - 1335 July 14-15, 2022 ©2022 Association for Computational Linguistics MT-Speech at SemEval-2022 Task 10: Incorporating Data Augmentation and Auxiliary Task with Cross-Lingual Pretrained Language Model for Structured Sentiment Analysis Cong Chen, Jiansong Chen, Cao Liu, Fan Yang, Guanglu Wan, Jinxiong Xia, Meituan, Beijing, China {chencong29, chenjiansong, liucao}@meituan.com Abstract Structured Sentiment Analysis (SSA) is an important component of sentiment analysis, which is a critical task in NLP. Traditional SSA methods cannot capture the cross-lingual inter- actions between different language and there is insufficient annotated data, especially in the cross-lingual settings. In this paper, we use the pre-trained language models with two auxiliary tasks and adopt data augmentation to address the above problems. Specifically, we employ XLM-RoBERTa to capture the cross-lingual knowledge interactions and enhance the gener- alization in multilingual/cross-lingual settings. Furthermore, we leverage two data augmenta- tion techniques and propose two auxiliary tasks to improve the performance on the few-shot and zero-shot settings. Experiments demonstrate that our model ranks first on the cross-lingual sub-task and second on the monolingual sub- task of SemEval-2022 task 10. 1 Introduction Structured Sentiment Analysis (SSA) is an impor- tant task in sentiment analysis (Barnes et al., 2021; Liu, 2012; Mitchell et al., 2013). The goal of SSA is to extract all opinion tuples from given texts. The opinion tuple (h, t, e, p) consists of a holder (h) which expresses a polarity (p) towards a target (t) by a textual sentiment expression (e). Benefiting a variety of business applications, such as human- machine dialogue and recommendation systems, SSA has attracted much more attention from both academia and industry (Pang et al., 2008; Mitchell et al., 2013; Xu et al., 2020; Ovrelid et al., 2020; Li et al., 2019). The mainstream method for SSA is to adopt a pipeline approach by separately performing the subtasks including holder extraction and target ex- traction. However, such methods can not capture dependencies of multiple sub-tasks. To address this problem, Barnes et al. (2021) leverages graph- based dependency parsing to capture the depen- dencies among opinion tuples, where sentiment holders, targets and expressions are the nodes, and the relations of them as the arcs. This model has obtained state-of-the-art performance on SSA. However, the aforementioned methods still suf- fer from some important issues. Firstly, the knowl- edge of the pre-trained language models (PLMs) has not been fully exploited. In fact, the cross- lingual PLMs contain rich knowledge of the inter- actions among different languages. Secondly, the above data-driven models rely on a large amount of annotation data, but there is insufficient or even no annotated data in the real scene. For example, in SemEval-2022 shared task 10 (Barnes et al., 2022), the MultiBEU (Barnes et al., 2018) dataset has only 1215 sentences and the MultiBCA (Barnes et al., 2018) dataset have only 1341 sentences, and there is no training data for the target language in the cross-lingual setting, which heavily hinders the performance on SSA. To address …

Content truncated to 500 words