ALANIS at SemEval-2018 Task 3: A Feature Engineering Approach to Irony Detection in English Tweets

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 507–511 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics ALANIS at SemEval-2018 Task 3: A Feature Engineering Approach to Irony Detection in English Tweets Kevin Swanberg‡ and Madiha Mirza† and Ted Pedersen† and Zhenduo Wang†∗ ‡Department of Writing Studies †Department of Computer Science University of Minnesota Duluth, MN 55812, USA {swanb034,mirz0022,tpederse,wang7211}@d.umn.edu Abstract This paper describes the ALANIS system that participated in Task 3 of SemEval-2018. We develop a system for detection of irony, as well as the detection of three types of irony: ver- bal polar irony, other verbal irony, and situa- tional irony. The system uses a logistic regres- sion model in subtask A and a voted classifier in subtask B, both of which rely on manually developed features to identify ironic tweets. ALANIS placed 34th of 43 systems in subtask A and 26th of 31 systems in subtask B. 1 Introduction With the invention and growth of various social networking sites, irony and other creative lin- guistic devices have become increasingly preva- lent in online content. Particularly when con- sidering microblogging platforms like Twitter, which encourage users to share their thoughts and opinions on a wide variety of topics, the use of irony can be extremely common. This can have strong implications for various prob- lems in natural language processing, which of- ten have difficulty in processing this ironic con- tent (e.g., (Liu, 2012; Ghosh and Veale, 2016; Maynard and Greenwood, 2014)), thus motivating the development of an accurate irony detection system. While irony has many varying definitions, it is defined by the SemEval task organizers as a trope or figurative language whose actual meaning dif- fers from what is literally enunciated. Our sys- tem, ALANIS (Automated Location and Naming of Ironic Sentences), uses a manually developed feature set and a logistic regression classifier for subtask A and a voted classifier for subtask B, achieving mean accuracies of .650 and .607 re- spectively on the training set and .512 and .434 ∗Authors are ordered alphabetically by their first name. respectively on the test set. The F1-scores are .469 and .276 on the test set. 2 Task Description SemEval Task 3 involves two subtasks. In sub- task A, a tweet simply must be identified as ironic or non-ironic. In subtask B, three types of ironic content must be individually differentiated from non-ironic content. These three types of irony are verbal polar irony, other verbal irony, and situa- tional irony. The task organizers provided us with a training set of 3,834 tweets for both subtasks. 3 ALANIS Our system, ALANIS, uses manually developed features indicative of ironic content, and passes a feature matrix for each tweet to a logistic regres- sion classifier in subtask A and a voted classifier system which employs a logistic regression, SVM, and Random Forest classifier in Subtask B. It use the scores from each of these classifiers to ”vote” on the correct label for a tweet. 3.1 Feature Selection We explored …

Content truncated to 500 words