Multi-task Learning of Negation and Speculation for Targeted Sentiment Classification

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2838–2869 June 6–11, 2021. ©2021 Association for Computational Linguistics 2838 Multi-task Learning of Negation and Speculation for Targeted Sentiment Classification Andrew Moore∗ School of Computing and Communications, Lancaster University a.moore@lancaster.ac.uk Jeremy Barnes University of Oslo Department of Informatics jeremycb@ifi.uio.no Abstract The majority of work in targeted sentiment analysis has concentrated on finding better methods to improve the overall results. Within this paper we show that these models are not robust to linguistic phenomena, specifically negation and speculation. In this paper, we propose a multi-task learning method to incorporate information from syntactic and semantic auxiliary tasks, including negation and speculation scope detection, to create English-language models that are more robust to these phenomena. Further we create two challenge datasets to evaluate model performance on negated and speculative samples. We find that multi-task models and transfer learning via language modelling can improve performance on these challenge datasets, but the overall performances indicate that there is still much room for improve- ment. We release both the datasets and the source code at https://github.com/ jerbarnes/multitask_negation_ for_targeted_sentiment. 1 Introduction Targeted sentiment analysis (TSA) involves jointly predicting entities which are the targets of an opin- ion, as well as the polarity expressed towards them (Mitchell et al., 2013). The TSA task, which is part of the larger set of fine-grained sentiment analysis tasks, can enable companies to provide better rec- ommendations (Bauman et al., 2017), as well as give digital humanities scholars a quantitative ap- proach to identifying how sentiment and emotions develop in literature (Alm et al., 2005; Kim and Klinger, 2019). Modelling TSA has moved from sequence la- beling using conditional random fields (CRFs) (Mitchell et al., 2013) or Recurrent Neural Net- works (RNN) (Zhang et al., 2015a; Katiyar and Cardie, 2016; Ma et al., 2018), to Transformer ∗The authors contributed equally. models (Hu et al., 2019). However, all these im- provements have concentrated on making the best of the relatively small task-specific datasets. As an- notation for fine-grained sentiment is difficult and often has low inter-annotator agreement (Wiebe et al., 2005; Øvrelid et al., 2020), this data tends to be small and of varying quality. This lack of high-quality training data prevents TSA models from learning complex, compositional linguistic phenomena. For sentence-level sentiment classi- fication, incorporating compositional information from relatively small amounts of negation or spec- ulation data improves both robustness and general performance (Councill et al., 2010; Cruz et al., 2016; Barnes et al., 2020). Furthermore, trans- fer learning via language-modelling also improves fine-grained sentiment analysis (Hu et al., 2019; Li et al., 2019b). In this paper, we wish to explore two research questions: 1. Does multi-task learning of negation and spec- ulation lead to more robust targeted sentiment models? 2. Does transfer learning based on language- modelling already incorporate this informa- tion in a way that is useful for targeted senti- ment models? We explore a multi-task learning (MTL) ap- proach …

Content truncated to 500 words