TV-AfD: An Imperative-Annotated Corpus from The Big Bang Theory and Wikipedia’s Articles for Deletion Discussions

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6542–6548 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6542 TV-AfD: An Imperative-Annotated Corpus from The Big Bang Theory and Wikipedia’s Articles for Deletion Discussions Yimin Xiao*1, Zong-Ying Slaton*2, Lu Xiao1 School of Information Studies1, College of Arts and Sciences2 Syracuse University Syracuse, NY, USA {yxiao39, zho100, lxiao04}@syr.edu *These authors contributed equally to this work. Abstract In this study, we created an imperative corpus with speech conversations from dialogues in The Big Bang Theory and with the written comments in Wikipedia’s Articles for Deletion discussions. For the TV show data, 59 episodes containing 25,076 statements are used. We manually annotated imperatives based on the annotation guideline adapted from Condoravdi and Lauer’s study (2012) and used the retrieved data to assess the performance of syntax-based classification rules. For the Wikipedia AfD comments data, we first developed and leveraged a syntax-based classifier to extract 10,624 statements that may be imperative, and we manually examined the statements and then identified true positives. With this corpus, we also examined the performance of the rule-based imperative detection tool. Our result shows different outcomes for speech (dialogue) and written data. The rule-based classification performs better in the written data in precision (0.80) compared to the speech data (0.44). Also, the rule-based classification has a low-performance overall for speech data with the precision of 0.44, recall of 0.41, and f-1 measure of 0.42. This finding implies the syntax-based model may need to be adjusted for a speech dataset because imperatives in oral communication have greater syntactic varieties and are highly context-dependent. Keywords: Imperative, Corpus, Speech Resources, Text Classification 1. Introduction Imperative sentences or clauses are those expressed to the addressee a speaker’s directive to follow, wishes or invitations of the future, or suggestions of action. Detecting imperatives in both oral and written communication helps to identify these speakers’ intentions and the extracted information can be then used to automatically display instructional information or generate answers to an inquiry. Personal digital assistant is one example that utilizes oral imperative. What a personal digital assistant mostly does is to provide information, send messages, make calls, schedule an appointment, open or close the applications, and activate specific functions in the device upon users’ verbal “request”. Nowadays, despite the fact that people rely on personal digital assistant more and more, they still have trouble detecting or processing imperatives when the users don’t use the expected forms, including cases where there is no verb at the beginning of the sentence or when the imperatives contains non-affixal negative markers like not, n’t, never, neither, nobody, no, none, nor, nothing, and nowhere. Personal digital assistants are still unable to detect imperatives like “No one wakes me up tomorrow morning.” or “I would prefer you not to play music.” when the users want to turn off the alarm clocks set for tomorrow or when the users want to pause the music playlist. Hence, in order to help …

Content truncated to 500 words