Irony Detector at SemEval-2018 Task 3: Irony Detection in English Tweets using Word Graph

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 581–586 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics Irony Detector at SemEval-2018 Task 3: Irony Detection in English Tweets using Word Graph Usman Ahmed, Lubna Zafar, Faiza Qayyum and Muhammad Arshad Islam Parallel Computing Network, Department of Computer Science, Capital University of Science and Technology, Islamabad, Pakistan. usmanahmed189@gmail.com, lubbnaa@gmail.com, faizaqayyum@cust.edu.pk, arshad.islam@cust.edu.pk. Abstract This paper describes the Irony detection sys- tem that participates in SemEval-2018 Task 3: Irony detection in English tweets. The system participated in the subtasks A and B. This paper discusses the results of our sys- tem in the development, evaluation and post evaluation. Each class in the dataset is repre- sented as directed unweighted graphs. Then, the comparison is carried out with each class graph which results in a vector. This vector is used as features by machine learning algo- rithm. The model is evaluated on a hold on strategy. The organizers randomly split 80% (3,833 instances) training set (provided to the participant in training their system) and test- ing set 20% (958 instances). The test set is reserved to evaluate the performance of par- ticipants systems. During the evaluation, our system ranked 23 in the Coda Lab result of the subtask A (binary class problem). The bi- nary class system achieves accuracy 0.6135, precision 0.5091, recall 0.7170 and F mea- sure 0.5955. The subtask B (multi-class prob- lem) system is ranked 22 in Coda Lab results. The multiclass model achieves the accuracy 0.4158, precision 0.4055, recall 0.3526 and f measure 0.3101. 1 Introduction Social media are deemed as a diverse web-based network that serves as an online platform to com- municate and disseminate information or ideas among individuals and fraternities. Since its ad- vent, people all around the globe harness it as a major source to express their opinions or emo- tions, however, an expeditious increase in its us- age has been reported in the last decade (Kelly et al., 2016; Perrin, 2015). Among the multifar- ious range of social media platforms, Twitter is the most popular one. It is basically a microblog- ging site diffuses information pertaining to what is happening around the world, and what are the cur- rent top-interest areas among the wider population (Rosenthal et al., 2017). According to a recent sur- vey, 6000 tweets per second are sent by 320 mil- lion active monthly users, thus 500 million tweets per day (Statistics, 2014). This poses a challenge for the scientific community to accurately discern the sentiment of a tweet out of this plethora. Since certain aspects associated with sentiment analysis are quite arduous yet feasible to ascertain (such as negative, positive, a neutral aspect of the opinion) than irony. Irony detection has its implications in senti- ment analysis (Reyes et al., 2009), opinion min- ing (Sarmento et al., 2009) and advertising (Kreuz, 2001). For the past few years, irony-aware sen- timent analysis has attained significant computa- tional treatment due to the prevalence of irony on the …

Content truncated to 500 words