Are Word Embedding-based Features Useful for Sarcasm Detection?

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1006–1011, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Are Word Embedding-based Features Useful for Sarcasm Detection? Aditya Joshi1,2,3 Vaibhav Tripathi1 Kevin Patel1 Pushpak Bhattacharyya1 Mark Carman2 1Indian Institute of Technology Bombay, India 2Monash University, Australia 3IITB-Monash Research Academy, India {adityaj,kevin.patel,pb}@cse.iitb.ac.in, mark.carman@monash.edu Abstract This paper makes a simple increment to state-of- the-art in sarcasm detection research. Existing ap- proaches are unable to capture subtle forms of con- text incongruity which lies at the heart of sarcasm. We explore if prior work can be enhanced using se- mantic similarity/discordance between word embed- dings. We augment word embedding-based features to four feature sets reported in the past. We also ex- periment with four types of word embeddings. We observe an improvement in sarcasm detection, irre- spective of the word embedding used or the original feature set to which our features are augmented. For example, this augmentation results in an improve- ment in F-score of around 4% for three out of these four feature sets, and a minor degradation in case of the fourth, when Word2Vec embeddings are used. Finally, a comparison of the four embeddings shows that Word2Vec and dependency weight-based fea- tures outperform LSA and GloVe, in terms of their benefit to sarcasm detection. 1 Introduction Sarcasm is a form of verbal irony that is intended to ex- press contempt or ridicule. Linguistic studies show that the notion of context incongruity is at the heart of sar- casm (Ivanko and Pexman, 2003). A popular trend in automatic sarcasm detection is semi-supervised extrac- tion of patterns that capture the underlying context in- congruity (Davidov et al., 2010; Joshi et al., 2015; Riloff et al., 2013). However, techniques to extract these pat- terns rely on sentiment-bearing words and may not cap- ture nuanced forms of sarcasm. Consider the sentence ‘With a sense of humor like that, you could make a liv- ing as a garbage man anywhere in the country.1’ The speaker makes a subtle, contemptuous remark about the 1All examples in this paper are actual instances from our dataset. sense of humor of the listener. However, absence of sen- timent words makes the sarcasm in this sentence difficult to capture as features for a classifier. In this paper, we explore use of word embeddings to capture context incongruity in the absence of sentiment words. The intuition is that word vector-based sim- ilarity/discordance is indicative of semantic similar- ity which in turn is a handle for context incongruity. In the case of the ‘sense of humor’ example above, the words ‘sense of humor’ and ‘garbage man’ are seman- tically dissimilar and their presence together in the sen- tence provides a clue to sarcasm. Hence, our set of fea- tures based on word embeddings aim to capture such se- mantic similarity/discordance. Since such semantic simi- larity is but one of the components of context incongruity and since existing feature sets rely on sentiment-based features to capture context incongruity, it …

Content truncated to 500 words