Hitachi at SemEval-2022 Task 10: Comparing Graph- and Seq2Seq-based Models Highlights Difficulty in Structured Sentiment Analysis

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1349 - 1359 July 14-15, 2022 ©2022 Association for Computational Linguistics Hitachi at SemEval-2022 Task 10: Comparing Graph- and Seq2Seq-based Models Highlights Difficulty in Structured Sentiment Analysis Gaku Morio, Hiroaki Ozaki, Atsuki Yamaguchi, and Yasuhiro Sogawa Research and Development Group, Hitachi, Ltd. Kokubunji, Tokyo, Japan {gaku.morio.vn, hiroaki.ozaki.yu, atsuki.yamaguchi.xn, yasuhiro.sogawa.tp}@hitachi.com Abstract This paper describes our participation in SemEval-2022 Task 10, a structured sentiment analysis. In this task, we have to parse opin- ions considering both structure- and context- dependent subjective aspects, which is differ- ent from typical dependency parsing. Some of the major parser types have recently been used for semantic and syntactic parsing, while it is still unknown which type can capture structured sentiments well due to their sub- jective aspects. To this end, we compared two different types of state-of-the-art parser, namely graph-based and seq2seq-based. Our in-depth analyses suggest that, even though graph-based parser generally outperforms the seq2seq-based one, with strong pre-trained language models both parsers can essentially output acceptable and reasonable predictions. The analyses highlight that the difficulty de- rived from subjective aspects in structured sen- timent analysis remains an essential challenge. 1 Introduction SemEval-2022 Task 10 (Barnes et al., 2022) aims at extracting structured sentiment from a given sentence. Different from other sentiment analy- sis tasks, structured sentiment analysis is formu- lated as an information extraction problem with at least three elements, namely a holder, a target and a sentiment expression. The shared task has two subtasks. In Subtask 1 (monolingual), we evaluate the performance on seven monolingual corpora, i.e., MPQA (Wiebe et al., 2005), OpeNER, OpeNER_es (Agerri et al., 2013), DSu (Toprak et al., 2010), MultiB_ca, MultiB_eu (Barnes et al., 2018) and NoReC (Øvrelid et al., 2020). In Subtask 2 (crosslingual), we evaluate the zero-shot predic- tion performance on three non-English corpora, i.e., OpeNER_es, MultiB_ca and MultiB_eu, generated by a model trained with an English cor- pus. One significant difference between syntactic parsing and structured sentiment analysis is that cannot open [Negative] windows [TGT] [SRC] No [Negative] balcony [TGT] [SRC] “No balcony, cannot open windows. “ No [0:2] balcony [3:10] cannot open [13:24] windows [25:32] Negative Negative Target Target Seq2Seq (T5) Graph parser (RoBERTa+ biaffine) Input text Serialized output Span aligner Output opinion1 Polarity = Negative Expression [0:2] Source = null Target=[3:10] Output opinion2 Polarity = Negative Expression=[13:24] Source = null Target=[25:32] Parsed graph Figure 1: Overview of the two parsers: (Top) Graph and (Bottom) Seq2Seq. the former is based on certain well-defined rules, whereas the latter forms an information structure on the basis of subjective opinions. Given the na- ture of the sentiment structure, Barnes et al. (2021) formulated the problem as a dependency parsing task. Our motivation here is to throw light upon how subjective opinions affect parsing performance and how well recent strong parsing models can cap- ture them. To this end, we compare two types of parsing model: graph- and generation-based1 models. Though these parsers have shown compet- itive performance under …

Content truncated to 500 words