KLUEnicorn at SemEval-2018 Task 3: A Naive Approach to Irony Detection

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 607–612 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics KLUEnicorn at SemEval-2018 Task 3: A Na¨ıve Approach to Irony Detection Luise D¨urlich Friedrich-Alexander Universit¨at Erlangen-N¨urnberg / Germany luise.duerlich@fau.de Abstract This paper describes the KLUEnicorn system submitted to the SemEval-2018 task on “Irony detection in English tweets”. The proposed system uses a na¨ıve Bayes classifier to exploit rather simple lexical, pragmatic and semantic features as well as sentiment. It further takes a closer look at different adverb categories and named entities and factors in word-embedding information. 1 Introduction Automatic irony and sarcasm detection has made great advances in recent years, evolving from considering purely lexical information (Kreuz and Caucci, 2007) to sentiment (Gonz´alez-Ib´a˜nez et al., 2011) and semantics (Ghosh et al., 2015). With new approaches that are aware of the con- text a tweet is produced in, promising results of as much as 87% accuracy (Silvio et al., 2016) have been achieved. In the following sections, I present a constrained contribution to the SemEval-2018 irony detection task (Van Hee et al., 2018). As useful context for the training data was rather hard to come by, a solely tweet based approach is explored. In the next section, the dataset provided by the task or- ganizers will be discussed. Sections 3 and 4 will elaborate on data preprocessing and the types of features that were tested. Finally, sections 5, 6 and 7 will present experiments on the usefulness of dif- ferent features to different classifiers, the settings used for the submitted systems and the competi- tion results. 2 Data To train the system, only the official training set consisting of 3,834 tweets was used. Of these tweets 1,911 were ironic and 1,923 were non- ironic. Depending on the subtask at hand, namely binary irony detection (task A) or the differenti- ation between different types of irony (task B), the ironic tweets were further categorized as ei- ther verbal irony by means of polarity contrast (class 1), other verbal irony (class 2) or situational irony (class 3). This resulted in 1,390 examples for class 1, 316 examples for class 2 and 205 ex- amples for class 3. The tweets still contained the original URLs, that were further analyzed to get an idea of whether or not they could provide use- ful context information. However, only as much as 14% of the ironic sample even contained URLs and most of these just linked to images and the original tweet on Twitter. As the data did not in- clude the names of the authors or contained any additional context information,1 context with re- spect to the authors user profile was not explored further. 3 Preprocessing As preparation for tagging, segmentation prob- lems – especially arising around emoji and punctu- ation marks – were corrected, user mentions were anonymized to “@user” and URLs replaced by “http://url.com”. Hashtags were stripped of the “#” and segmented using a simple hand-crafted hashtag tokenizer that relies on …

Content truncated to 500 words