MAD-TSC: A Multilingual Aligned News Dataset for Target-dependent Sentiment Classification

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8286–8305 July 9-14, 2023 ©2023 Association for Computational Linguistics MAD-TSC: A Multilingual Aligned News Dataset for Target-dependent Sentiment Classification Evan Dufraisse†*, Adrian Popescu†, Julien Tourille†, Armelle Brun*, Jerome Deshayes† † Université Paris-Saclay, CEA, List, F-91120, Palaiseau, France * Université de Lorraine - CNRS - Loria, Vandoeuvre les Nancy Cedex, France Abstract Target-dependent sentiment classifica- tion (TSC) enables a fine-grained automatic analysis of sentiments expressed in texts. Sentiment expression varies depending on the domain, and it is necessary to create domain-specific datasets. While socially important, TSC in the news domain remains relatively understudied. We introduce MAD- TSC, the first multilingual aligned dataset designed for TSC in news. MAD-TSC differs substantially from existing resources. First, it includes aligned examples in eight languages to facilitate a comparison of performance for individual languages, and a direct comparison of human and machine translation. Second, the dataset is sampled from a diversified parallel news corpus, and is diversified in terms of news sources and geographic spread of entities. Finally, MAD-TSC is more challenging than existing datasets because its samples are more complex. We exemplify the use of MAD-TSC with comprehensive monolingual and multilingual experiments. The latter shows that machine translations can successfully replace manual ones, and that performance for all included languages can match that of English by automatically translating test examples. 1 Introduction Text analysis needs to address both objective as- pects, such as topic extraction, and subjective as- pects, such as sentiment and opinion classification. In spite of recent progress brought by the intro- duction of large language models (Devlin et al., 2019; Liu et al., 2019; Yang et al., 2019), senti- ment classification remains a challenging task. Ex- pression of sentiments varies according to the data sources, languages and domain of the texts. These challenges are particularly important in target- dependent sentiment classification (TSC), which focuses on determining the sentiment expressed to- ward a given entity in a given context. The bulk of TSC-related research efforts are focused on major languages. This focus is an effect of the availability of generic and of task-specific resources in these languages (Brauwers and Frasincar, 2022; Nazir et al., 2020). A majority of datasets are mono- lingual and, when they are multilingual (Balahur and Turchi, 2014; Barriere and Balahur, 2020; Cor- tis and Davis, 2021; Pontiki et al., 2016; Severyn et al., 2016), the examples are not aligned across languages. Equally, a majority of existing datasets and methods are devised for texts such as tweets, reviews or comments (Nakov et al., 2016; Pontiki et al., 2016; Severyn et al., 2016) in which senti- ment is most often expressed explicitly. Somewhat surprisingly, less attention is given to TSC in news, despite the usefulness of an automatic analysis of their content for the understanding of societally impactful processes such as disinformation or po- larization (Hamborg and Donnay, 2021). Our main contribution is the introduction of MAD-TSC, the first large multilingual aligned dataset …

Content truncated to 500 words