Tackling Irony Detection using Ensemble Classifiers

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6976–6984 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6976 Tackling Irony Detection using Ensemble Classifiers and Data Augmentation Christoph Turban, Udo Kruschwitz University of Regensburg christoph.turban@gmail.com, udo.kruschwitz@ur.de Abstract Automatic approaches to irony detection have been of interest to the NLP community for a long time, yet, state-of-the-art approaches still fall way short of what one would consider a desirable performance. In part this is due to the inherent difficulty of the problem. However, in recent years ensembles of transformer-based approaches have emerged as a promising direction to push the state of the art forward in a wide range of NLP applications. A different, more recent, development is the automatic augmentation of training data. In this paper we will explore both these directions for the task of irony detection in social media. Using the common SemEval 2018 Task 3 benchmark collection we demonstrate that transformer models are well suited in ensemble classifiers for the task at hand. In the multi-class classification task we observe statistically significant improvements over strong baselines. For binary classification we achieve performance that is on par with state-of-the-art alternatives. The examined data augmentation strategies showed an effect, but are not decisive for good results. Keywords: Irony detection, Ensembles, Transformers, BERT, Data augmentation 1. Motivation Working with figurative language has long been con- sidered one of the most difficult topics in Natural Lan- guage Processing (NLP), e.g. (Reyes et al., 2012). In contrast to literal language, this type of language uses stylistic devices such as metaphors, rhetorical ques- tions as well as irony. For its understanding, a reader must use more abstract thinking and interpret informa- tion beyond syntax and semantics because the informa- tion needed for understanding is not expressed in the text itself (Reyes et al., 2012). If the underlying infor- mation is not unveiled, the actual meaning stays hidden and the speaker might be misunderstood. The detection of figurative language is particularly important in cases where irony is present because the actual meaning is often the opposite of the literal (Wilson and Sperber, 1992). The interpretation of irony can in fact be traced back all the way to Cicero’s De Oratore.1 Take the expression ”Great game” (assuming this has been said by a keen supporter of a sports team that has just lost a match). This is an example of irony where the speaker says something positive to express a neg- ative sentiment. What makes this difficult is that the meaning is not expressed with adjectives like ’terrible’, but instead, the polarity of the word ’great’ must be reversed. In order to know that ’great’ must not be in- terpreted in a literal way, the interpreter has to know the broader context. One might argue that this is not much of a problem in social chit-chat, it turns out that the problem has much wider implications as irony is widespread in abusive content, and therefore sturdy …

Content truncated to 500 words