DataStories at SemEval-2017 Task 6: Siamese LSTM with Attention for Humorous Text Comparison

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 390–395, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics DataStories at SemEval-2017 Task 6: Siamese LSTM with Attention for Humorous Text Comparison Christos Baziotis, Nikos Pelekis, Christos Doulkeridis University of Piraeus - Data Science Lab Piraeus, Greece mpsp14057@unipi.gr, npelekis@unipi.gr, cdoulk@unipi.gr Abstract In this paper we present a deep-learning system that competed at SemEval-2017 Task 6 “#HashtagWars: Learning a Sense of Humor”. We participated in Subtask A, in which the goal was, given two Twitter messages, to identify which one is fun- nier. We propose a Siamese architecture with bidirectional Long Short-Term Mem- ory (LSTM) networks, augmented with an attention mechanism. Our system works on the token-level, leveraging word em- beddings trained on a big collection of un- labeled Twitter messages. We ranked 2nd in 7 teams. A post-completion improve- ment of our model, achieves state-of-the- art results on #HashtagWars dataset. 1 Introduction Computational humor (Stock and Strapparava, 2003) is an area in computational linguistics and natural language understanding. Most computa- tional humor tasks focus on the problem of humor detection. However SemEval-2017 Task 6 (Potash et al., 2017) explores the subjective nature of hu- mor, using a dataset of Twitter messages posted in the context of the TV show “@midnight”. At each episode during the segment “Hashtag Wars”, a topic in the form of a hashtag is given and view- ers of the show post funny tweets including that hashtag. In the next episode, the show selects the ten funniest tweets and a final winning tweet. In the past, computational humor tasks have been approached using hand-crafted features (Hempelmann, 2008; Mihalcea and Strapparava, 2006; Kiddon and Brun, 2011; Yang et al., 2015). However, these approaches require a laborious feature-engineering process, which usually leads to missing or redundant features, especially in the case of humor, which is hard to define and con- sequently hard to model. Recently, approaches us- ing neural networks, that perform feature-learning, have shown great results (Chen and Lee, 2017; Potash et al., 2016; Bertero and Fung, 2016a,b) outperforming the traditional methods. In this paper, we present a deep-learning system that we developed for subtask A - “Pairwise Com- parison”. The goal of the task is, given two tweets about the same topic, to identify which one is fun- nier. The labels are applied using the show’s rel- ative ranking. This is a very challenging task, be- cause humor is subjective and the machine learn- ing system must develop a sense of humor similar to that of the show, in order to perform well. We employ a Siamese neural network, which generates a dense vector representation for each tweet and then uses those representations as fea- tures for classification. For modeling the Twit- ter messages we use Long Short-Term Mem- ory (LSTM) networks augmented with a context- aware attention mechanism (Yang et al., 2016). Furthermore, we perform thorough text prepro- cessing that enables our neural network to learn better features. Finally, our approach …

Content truncated to 500 words