WADER at SemEval-2023 Task 9: A Weak-labelling framework for Data augmentation in tExt Regression Tasks

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1945–1952 July 13-14, 2023 ©2023 Association for Computational Linguistics WADER at SemEval-2023 Task 9: A Weak-labelling framework for Data augmentation in tExt Regression Tasks Manan Suri1∗, Aaryak Garg1*, Divya Chaudhary2, Ian Gorton2, Bijendra Kumar1 Netaji Subhas University of Technology, New Delhi1 Northeastern University, Seattle 2 {manan.suri.ug20,aaryak.ug20}@nsut.ac.in Abstract Intimacy is an essential element of human re- lationships and language is a crucial means of conveying it. Textual intimacy analysis can reveal social norms in different contexts and serve as a benchmark for testing computational models’ ability to understand social informa- tion. In this paper, we propose a novel weak- labeling strategy for data augmentation in text regression tasks called WADER. WADER uses data augmentation to address the problems of data imbalance and data scarcity and provides a method for data augmentation in cross-lingual, zero-shot tasks. We benchmark the perfor- mance of State-of-the-Art pre-trained multi- lingual language models using WADER and analyze the use of sampling techniques to miti- gate bias in data and optimally select augmenta- tion candidates. Our results show that WADER outperforms the baseline model and provides a direction for mitigating data imbalance and scarcity in text regression tasks. 1 Introduction Intimacy is considered a fundamental element of human relationships, as recognized by several scholars (Sullivan, 1993; Maslow, 1981; Prager, 1995). Research indicates that intimacy can be modeled computationally and that textual intimacy is a crucial aspect of language (Pei and Jurgens, 2020). Analyzing textual intimacy can reveal social norms in various contexts and serve as a benchmark to test computational models’ ability to understand social information (Pei and Jurgens, 2020; Hovy and Yang, 2021). Moreover, intimacy plays a crit- ical role in human development and well-being (Harlow and Zimmermann, 1959; Sneed et al., 2011), and language is an essential means of con- veying it in a social context. Individuals nego- tiate intimacy in language to fulfill fundamental and strategic needs while respecting social norms. ∗*Equal contribution. Task 9 of SemEval 2023 (Pei et al., 2022) aims to quantify intimacy in a multilingual context, with evaluation on tweets from 10 languages. The train- ing corpus for the task consists of tweets in En- glish, Spanish, Italian, Portuguese, French, and Chinese. The testing corpus additionally contains tweets from Hindi, Arabic, Dutch and Korean. The novelty of our strategy, WADER (Weak- labeling strategy for Data augmentation in tExt Regression Tasks) is the use of data augmentation to A) solve the problem of an imbalance distribu- tion of data, B) augment data for a cross-lingual zero-shot set-up. WADER uses the distribution to selectively sample texts with lower representation in the label distribution, uses translation to augment sentences and validates the augmentations against a baseline model, using a distribution based sampling approach. We finetune State-of-the-Art pre-trained language models including XLM RoBERTa (Con- neau et al., 2019) and XLNET (Yang et al., 2019). Real world datasets are plagued by the problems of data imbalance and data scarcity, and WADER pro- vides a …

Content truncated to 500 words