SLPL-Sentiment at SemEval-2022 Task 10: Making Use of Pre-Trained Model’s Attention Values in Structured Sentiment Analysis

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1382 - 1388 July 14-15, 2022 ©2022 Association for Computational Linguistics SLPL-Sentiment at SemEval-2022 Task 10: Making Use of Pre-Trained Model’s Attention Values in Structured Sentiment Analysis Sadrodin Barikbin Speech and Language Processing Lab Sharif University of Technology barikbin@ce.sharif.edu Abstract Sentiment analysis is a useful problem which could serve a variety of fields from business intelligence to social studies and even health studies. Using SemEval 2022 Task 10 formu- lation of this problem and taking sequence la- belling as our approach, we propose a model which learns the task by finetuning a pre- trained transformer, introducing as few param- eters ( 150k) as possible and making use of pre- computed attention values in the transformer. Our model improves shared task baselines on all task datasets. 1 Introduction Sentiment analysis has many applications in dif- ferent fields. From social and political studies (Rodr´ıguez-Ib´a˜nez et al., 2021) to business intel- ligence and even health studies (Alamoodi et al., 2021). SemEval 2022 Task 10 (Barnes et al., 2022) formulates this problem as to extract a graph of sentiment-related entities and names that, Struc- tured Sentiment Analysis. An example of such graph is depicted in figure2 Specifically given an in- put sentence, one should extract a list of quadruples. Each quadruple consists of a sentiment expression, a target expression, a holder expression and the polarity. SemEval 2022 Task 10 has two subtasks, monolingual and cross-lingual. We participated in the first track. Shared task data is consisted of seven datasets in English, Spanish, Norwegian, Catalan and Basque. We trained and evaluated our model on each of these datasets.1 2 Related Work Structured sentiment analysis aims to extract sentiment, target and holder expressions along 1Our code is available at https: //github.com/sadra-barikbin/ novel-solutions-for-sentiment-analysis with their relations and also sentiment polarities. (Barnes et al., 2021) solves this problem by taking dependency graph parsing approach. It uses BiL- STM and multilingual BERT (Devlin et al., 2019) to encode input sentence and a neural dependency parser to jointly predict expressions and relations. Every sentiment, target or holder expression in the sentence would become a star-shaped subtree in the output graph, with last token of expression as its root and its edges labelled with type of entity. Relation between a sentiment expression and a tar- get or holder expression is also represented with an edge from the former’s root to that of the latter. This work evaluates its model on five datasets in four languages namely, NoReCFine (Øvrelid et al., 2020), MultibookedEU, MultibookedCA (Barnes et al., 2018), MPQA (Wiebe et al., 2005) and DarmstadtUnis (Toprak et al., 2010). Its main evalu- ation metrics are Targeted F1 and Sentiment Graph F1. First one measures exact prediction of target ex- pression and polarity, and the second one measures exact match at graph level, weighting overlap be- tween gold and predicted entities, averaged across three entity types. A less comprehensive formulation of sentiment analysis is End2End sentiment analysis which aims to predict target …

Content truncated to 500 words