NIHRIO at SemEval-2018 Task 3: A Simple and Accurate Neural Network Model for Irony Detection in Twitter

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 525–530 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics NIHRIO at SemEval-2018 Task 3: A Simple and Accurate Neural Network Model for Irony Detection in Twitter Thanh Vu1, Dat Quoc Nguyen2, Xuan-Son Vu3, Dai Quoc Nguyen4, Michael Catt1 and Michael Trenell1 1NIHRIO, Newcastle University, UK; 2The University of Melbourne, Australia; 3Ume˚a University, Sweden; 4Deakin University, Australia thanh.vu@io.nihr.ac.uk; dqnguyen@unimelb.edu.au; sonvx@cs.umu.se; dai.nguyen@deakin.edu.au; {michael.catt, michael.trenell}@io.nihr.ac.uk Abstract This paper describes our NIHRIO system for SemEval-2018 Task 3 “Irony detection in En- glish tweets.” We propose to use a simple neu- ral network architecture of Multilayer Percep- tron with various types of input features in- cluding: lexical, syntactic, semantic and po- larity features. Our system achieves very high performance in both subtasks of binary and multi-class irony detection in tweets. In par- ticular, we rank third using the accuracy met- ric and fifth using the F1 metric. Our code is available at: https://github.com/ NIHRIO/IronyDetectionInTwitter. 1 Introduction Mining Twitter data has increasingly been attract- ing much research attention in many NLP appli- cations such as in sentiment analysis (Pak and Paroubek, 2010; Kouloumpis et al., 2011; Agar- wal et al., 2011; Liu et al., 2012; Rosenthal et al., 2017; Cambria et al., 2018) and stock market pre- diction (Bollen et al., 2011; Vu et al., 2012; Bartov et al., 2015; Nofer and Hinz, 2015; Oliveira et al., 2017). Recently, Davidov et al. (2010) and Reyes et al. (2013) have shown that Twitter data includes a high volume of “ironic” tweets. For example, a user can use positive words in a Twitter mes- sage to her intended negative meaning (e.g., “It is awesome to go to bed at 3 am #not”). This es- pecially results in a research challenge to assign correct sentiment labels for ironic tweets (Bosco et al., 2013; Ghosh et al., 2015; Far´ıas et al., 2016; Nozza et al., 2017; Kannangara, 2018). To handle that problem, much attention has been focused on automatic irony detection in Twit- ter (Davidov et al., 2010; Reyes et al., 2013; Barbi- eri and Saggion, 2014; Rajadesingan et al., 2015; Far´ıas et al., 2016; Sulis et al., 2016; Karoui et al., 2017; Joshi et al., 2017; Huang et al., 2017; Ravi and Ravi, 2017). In this paper, we propose a neural network model for irony detection in tweets. Our model obtains the fifth best performances in both binary and multi-class irony detection subtasks in terms of F1 score (Van Hee et al., 2018). Details of the two subtasks can be found in the task de- scription paper (Van Hee et al., 2018). We briefly describe the subtasks as follows: Subtask 1 (A): Ironic vs non-ironic This first subtask is a binary classification problem, in which we predict whether or not a tweet is ironic. For example, “I just love when you test my pa- tience!! #not” is ironic, but “Had no sleep and have got school now #not happy” is non-ironic. Subtask 2 (B): Different …

Content truncated to 500 words