Harnessing Sequence Labeling for Sarcasm Detection in Dialogue from TV Series ‘Friends’

Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), pages 146–155, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Harnessing Sequence Labeling for Sarcasm Detection in Dialogue from TV Series ‘Friends’ Aditya Joshi1,2,3 Vaibhav Tripathi1 Pushpak Bhattacharyya1 Mark Carman2 1Indian Institute of Technology Bombay, India 2Monash University, Australia 3IITB-Monash Research Academy, India {adityaj, vtripathi,pb}@cse.iitb.ac.in, mark.carman@monash.edu Abstract This paper is a novel study that views sarcasm detection in dialogue as a sequence labeling task, where a dialogue is made up of a se- quence of utterances. We create a manually- labeled dataset of dialogue from TV series ‘Friends’ annotated with sarcasm. Our goal is to predict sarcasm in each utterance, using sequential nature of a scene. We show perfor- mance gain using sequence labeling as com- pared to classification-based approaches. Our experiments are based on three sets of features, one is derived from information in our dataset, the other two are from past works. Two sequence labeling algorithms (SVM-HMM and SEARN) outperform three classification algorithms (SVM, Naive Bayes) for all these feature sets, with an increase in F-score of around 4%. Our observations high- light the viability of sequence labeling tech- niques for sarcasm detection of dialogue. 1 Introduction Sarcasm is defined as ‘the use of irony to mock or con- vey contempt’1. An example of a sarcastic sentence is ‘Being stranded in traffic is the best way to start the week’. In this case, the positive word ‘best’ together with the undesirable situation ‘being stranded in traf- fic’ conveys the sarcasm. Because sarcasm has an im- plied sentiment (negative) that is different from surface sentiment (positive due to presence of ‘best’), it poses a challenge to sentiment analysis systems that aim to determine polarity in text (Pang and Lee, 2008). Some sarcastic expressions may be more difficult to detect. Consider the possibly sarcastic statement ‘I ab- solutely love this restaurant’. Unlike in the traffic ex- ample above, sarcasm in this sentence, if any, can be understood using context which is ‘external’ to the sen- tence i.e., beyond common world knowledge. 2. This external context may be available in the conversation 1As defined by the Oxford Dictionary. 2Common world knowledge here refers to a general sen- timent map of situations to sentiment. For example, being stranded in traffic is a negative situation to most. that this sentence is a part of. For example, the con- versational context may be situational: the speaker dis- covers a fly in her soup, then looks at her date and says, ‘I absolutely love this restaurant’. The conversational context may also be verbal: her date says, ‘They’ve taken 40 minutes to bring our appetizers’ to which the speaker responds ‘I absolutely love this restaurant’. Both these examples point to the intuition that for di- alogue (i.e., data where more than one speaker partici- pates in a discourse), conversational context is often a clue for sarcasm. For such dialogue, prior work in sarcasm detection (determining whether a text is sarcastic or not) …

Content truncated to 500 words