Random Decision Syntax Trees at SemEval-2018 Task 3: LSTMs and Sentiment Scores for Irony Detection

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 560–564 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics Random Decision Syntax Trees at SemEval-2018 Task 3: LSTMs and Sentiment Scores for Irony Detection Aidan San University of Illinois at Urbana-Champaign asan2@illinois.edu Abstract We propose a Long Short Term Memory Neu- ral Network model for irony detection in tweets in this paper. Our model is trained us- ing word embeddings and emoji embeddings. We show that adding sentiment scores to our model improves the F1 score of our baseline LSTM by approximately .012, and therefore show that high-level features can be used to improve word embeddings in certain Natural Language Processing applications. Our model ranks 24/43 for binary classification and 5/31 for multiclass classification. We make our model easily accessible to the research com- munity1. 1 Introduction Recently, irony detection has become an increas- ingly important problem with new applications ap- pearing every day. In discourse analysis, it is ex- tremely important to understand if a politician is making an ironic or a literal response, or under- standing can be completely lost. In chatbots, if a chatbot misinterprets an unhappy sarcastic com- ment from a customer, the customer could become even more frustrated. In sentiment analysis, if irony is not taken into account, the actual senti- ment could be the opposite of the prediction. The aim of irony detection is generally a binary classfication problem: Is the piece of text ironic or literal? In SemEval2018 Task 3 (Van Hee et al., 2018), there are two subtasks. First, subtask A, is our standard binary classification problem. We are provided with a corpus of tweets annotated with 0 or 1’s to specify if a tweet is ironic or literal. Second, in subtask B, we are tasked with a more challenging problem. In subtask B, participants must determine which type of irony a particular tweet contains. Is it verbal irony based on polarity, 1github.com/Muakasan/RDST-semeval2018-task3 another type of verbal irony, situational irony or not ironic at all? Recently, research has shown that neural ap- proaches are particularly effective in sarcasm de- tection (Ghosh and Veale, 2016) and similar prob- lems such as sentiment analysis (Rosenthal et al., 2017). In this paper, we present neural network model designed to tackle the challenge of irony detection. In subsection 2.1, we give a brief overview of the entire model. In subsection 2.2 and 2.3, we describe our neural architecture and approach. In subsection 2.5 and 2.6, we describe the embed- dings and sentiment scores that are used as the in- put to our neural network. In section 3, we de- scribe our results, and give quantitative evidence of the effectiveness of our features. In section 4 we conclude, and in section 5, we describe ways that our approach could be improved. 2 System Description 2.1 Overview Our system is composed of three stages: Prepro- cessing, Feature Creation, and the Neural Net- work. Preprocessing is composed of tokenizing the input text. Feature creation is …

Content truncated to 500 words