Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset
8750Authors
nan
Abstract
AbstractWe introduce the well-established social scientific concept of social solidarity and its contestation, anti-solidarity, as a new problem setting to supervised machine learning in NLP to assess how European solidarity discourses changed before and after the COVID-19 outbreak was declared a global pandemic. To this end, we annotate 2.3k English and German tweets for (anti-)solidarity expressions, utilizing multiple human annotators and two annotation approaches (experts vs. crowds). We use these annotations to train a BERT model with multiple data augmentation strategies. Our augmented BERT model that combines both expert and crowd annotations outperforms the baseline BERT classifier trained with expert annotations only by over 25 points, from 58% macro-F1 to almost 85%. We use this high-quality model to automatically label over 270k tweets between September 2019 and December 2020. We then assess the automatically labeled data for how statements related to European (anti-)solidarity discourses developed over time and in relation to one another, before and during the COVID-19 crisis. Our results show that solidarity became increasingly salient and contested during the crisis. While the number of solidarity tweets remained on a higher level and dominated the discourse in the scrutinized time frame, anti-solidarity tweets initially spiked, then decreased to (almost) pre-COVID-19 values before rising to a stable higher level until the end of 2020.
Keywords
Full Content
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 139–151 August 1–6, 2021. ©2021 Association for Computational Linguistics 139 Dual Slot Selector via Local Reliability Verification for Dialogue State Tracking Jinyu Guo1, Kai Shuang1∗, Jijie Li1 and Zihan Wang2 1State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications 2Graduate School of Information Science and Technology, The University of Tokyo {guojinyu, shuangk, lijijie}@bupt.edu.cn zwang@tkl.iis.u-tokyo.ac.jp Abstract The goal of dialogue state tracking (DST) is to predict the current dialogue state given all pre- vious dialogue contexts. Existing approaches generally predict the dialogue state at every turn from scratch. However, the overwhelm- ing majority of the slots in each turn should simply inherit the slot values from the previ- ous turn. Therefore, the mechanism of treat- ing slots equally in each turn not only is in- efficient but also may lead to additional er- rors because of the redundant slot value gen- eration. To address this problem, we devise the two-stage DSS-DST which consists of the Dual Slot Selector based on the current turn dialogue, and the Slot Value Generator based on the dialogue history. The Dual Slot Selec- tor determines each slot whether to update slot value or to inherit the slot value from the pre- vious turn from two aspects: (1) if there is a strong relationship between it and the cur- rent turn dialogue utterances; (2) if a slot value with high reliability can be obtained for it through the current turn dialogue. The slots selected to be updated are permitted to enter the Slot Value Generator to update values by a hybrid method, while the other slots directly inherit the values from the previous turn. Em- pirical results show that our method achieves 56.93%, 60.73%, and 58.04% joint accuracy on MultiWOZ 2.0, MultiWOZ 2.1, and Multi- WOZ 2.2 datasets respectively and achieves a new state-of-the-art performance with signifi- cant improvements. 1 1 Introduction Task-oriented dialogue has attracted increasing at- tention in both the research and industry communi- ties. As a key component in task-oriented dialogue systems, Dialogue State Tracking (DST) aims to ∗Corresponding author. 1Code is available at https://github.com/guojinyu88/DSSDST extract user goals or intents and represent them as a compact dialogue state in the form of slot-value pairs of each turn dialogue. DST is an essential part of dialogue management in task-oriented dialogue systems, where the next dialogue system action is selected based on the current dialogue state. Early dialogue state tracking approaches ex- tract value for each slot predefined in a single domain (Williams et al., 2014; Henderson et al., 2014a,b). These methods can be directly adapted to multi-domain conversations by replacing slots in a single domain with domain-slot pairs predefined. In multi-domain DST, some of the previous works study the scalability of the model (Wu et al., 2019), some aim to fully utilizing the dialogue history and context (Shan et al., 2020; Chen et al., 2020a; Quan and …
Content truncated to 500 words
Labeled Text
AbstractWe introduce the well-established social scientific concept of social solidarity and its contestation, anti-solidarity, as a new problem setting to supervised machine learning in NLP to assess how European solidarity discourses changed before and after the COVID-19 outbreak was declared a global pandemic. To this end, we annotate 2.3k English and German tweets for (anti-)solidarity expressions, utilizing multiple human annotators and two annotation approaches (experts vs. crowds). We use these annotations to train a BERT model with multiple data augmentation strategies. Our augmented BERT model that combines both expert and crowd annotations outperforms the baseline BERT classifier trained with expert annotations only by over 25 points, from 58% macro-F1 to almost 85%. We use this high-quality model to automatically label over 270k tweets between September 2019 and December 2020. We then assess the automatically labeled data for how statements related to European (anti-)solidarity discourses developed over time and in relation to one another, before and during the COVID-19 crisis. Our results show that solidarity became increasingly salient and contested during the crisis. While the number of solidarity tweets remained on a higher level and dominated the discourse in the scrutinized time frame, anti-solidarity tweets initially spiked, then decreased to (almost) pre-COVID-19 values before rising to a stable higher level until the end of 2020.
Publication Details
- Paper ID:
- 8750
- Year:
- 2021
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan