Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5394–5407 June 6–11, 2021. ©2021 Association for Computational Linguistics 5394 Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection Xu Guo, Boyang Li*, Han Yu and Chunyan Miao* School of Computer Science and Engineering, Nanyang Technological University, Singapore {xu008, han.yu, boyang.li, ascymiao}@ntu.edu.sg Abstract The existence of multiple datasets for sarcasm detection prompts us to apply transfer learn- ing to exploit their commonality. The adver- sarial neural transfer (ANT) framework uti- lizes multiple loss terms that encourage the source-domain and the target-domain feature distributions to be similar while optimizing for domain-specific performance. However, these objectives may be in conflict, which can lead to optimization difficulties and some- times diminished transfer. We propose a gen- eralized latent optimization strategy that al- lows different losses to accommodate each other and improves training dynamics. The proposed method outperforms transfer learn- ing and meta-learning baselines. In particular, we achieve 10.02% absolute performance gain over the previous state of the art on the iSar- casm dataset. 1 Introduction Sarcastic language is commonly found in social media posts (González-Ibáñez et al., 2011; May- nard and Greenwood, 2014), forum discussions (Khodak et al., 2018a), product reviews (Davidov et al., 2010; Filatova, 2012) and everyday conversa- tions (Gibbs, 2000). Detecting sarcasm is an inte- gral part of creative language understanding (Veale et al., 2019) and online opinion mining (Kannan- gara, 2018). Due to highly contextualized expres- sions, detecting sarcasm is a challenging task, even for humans (Fox Tree et al., 2020). A challenge specific to sarcasm detection is the difficulty in acquiring ground-truth annotations. Human-annotated datasets (Filatova, 2012; Riloff et al., 2013; Van Hee et al., 2018; Oprea and Magdy, 2020) usually contain only a few thousand texts, resulting in many small datasets. In comparison, automatic data collection using distant supervision signals like hashtags (Ptáˇcek et al., 2014; Bam- man and Smith, 2015; Joshi et al., 2015) yielded * Corresponding authors substantially larger datasets. Nevertheless, the auto- matic approach also led to label noise. For example, Oprea and Magdy (2020) found nearly half of the tweets with sarcasm hashtags in one dataset are not sarcastic. The existence of diverse datasets and data collec- tion methods prompts us to exploit their common- ality through transfer learning. Specifically, we transfer knowledge learned from large and noisy datasets to improve sarcasm detection on small human-annotated datasets that serve as effective performance benchmarks. Adversarial neural transfer (ANT) (Ganin and Lempitsky, 2015; Liu et al., 2017; Kim et al., 2017; Kamath et al., 2019) employs an adversarial setup where the network learns to make the shared feature distributions of the source domain and the target do- main as similar as possible, while simultaneously optimizing for domain-specific performance. How- ever, as the domain-specific losses promote the use of domain-specific features, these training objec- tives may compete with each other implicitly. This leads to optimization difficulties and potentially degenerate cases where the domain-specific classi- fiers ignore the shared …

Content truncated to 500 words