HashCount at SemEval-2018 Task 3: Concatenative Featurization of Tweet and Hashtags for Irony Detection

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 546–552 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics HashCount at SemEval-2018 Task 3: Concatenative Featurization of Tweet and Hashtags for Irony Detection Won Ik Cho, Woo Hyun Kang, Nam Soo Kim Department of Electrical and Computer Engineering and INMC, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, Korea, 08826 {wicho, whkang}@hi.snu.ac.kr, nkim@snu.ac.kr Abstract This paper proposes a novel feature extraction process for SemEval task 3: Irony detection in English tweets. The proposed system incor- porates a concatenative featurization of tweet and hashtags, which helps distinguishing be- tween the irony-related and the other compo- nents. The system embeds tweets into a vec- tor sequence with widely used pretrained word vectors, partially using a character embedding for the words that are out of vocabulary. Iden- tification was performed with BiLSTM and CNN classifiers, achieving F1 score of 0.5939 (23/42) and 0.3925 (10/28) each for the binary and the multi-class case, respectively. The re- liability of the proposed scheme was verified by analyzing the Gold test data, which demon- strates how hashtags can be taken into account when identifying various types of irony. 1 Introduction Nowadays, opinion mining from social media has become an important issue in natural language processing (NLP). Since tweets are globally used social media text that can influence the worldwide readers with just a short arrangement of words, the analysis on tweets has been widely studied in the semantic aspects such as sentiment classifica- tion (Rosenthal et al., 2017), hate speech detection (Waseem and Hovy, 2016), and irony detection (Karoui et al., 2015; Liebrecht et al., 2013). Espe- cially, the automatic detection of ironic tweets can help the readers who are having difficulty recog- nizing sarcasm to notice such figurative instances from excessive amount of text data. Despite the potential usage of the tasks, irony and sarcasm are difficult to grasp simply by an- alyzing word distribution. They require under- standings on the language and social context, which are dependent on time and space; this im- plies that the study should accompany constant up- dates on the database used for the analysis. Also, it is important to construct a concrete criteria set to distinguish between ironic and non-ironic tweets. In this paper, we incorporate a classification on manually labeled irony tweet corpus. The cor- pus contains 2,396 English tweets for ironic/non- ironic each (4,792 total), which were annotated under the scheme suggested in Van Hee et al. (2016). For the competition, the corpus was split into a training (80%, or 3,834 instances) and test (20%, or 958 instances) set. A training procedure for the system was all performed with the former (constrained). 2 Task Description Two tasks were proposed for ironic English tweet analysis. Task A deals with the binary classifica- tion involving ironic tweets and non-ironic ones, and in Task B the ironic ones are categorized into three types (i.e. verbal irony by means of a polar- ity contrast, other types of verbal irony, …

Content truncated to 500 words