Annotation and Evaluation of Coreference Resolution in Screenplays

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2004–2010 August 1–6, 2021. ©2021 Association for Computational Linguistics 2004 Annotation and Evaluation of Coreference Resolution in Screenplays Sabyasachee Baruah Sandeep Nallan Chakravarthula Shrikanth Narayanan sbaruah@usc.edu nallanch@usc.edu shri@ee.usc.edu Signal Analysis and Interpretation Laboratory University of Southern California Abstract Screenplays refer to characters using differ- ent names, pronouns, and nominal expres- sions. We need to resolve these mentions to the correct referent character for better story understanding and holistic research in compu- tational narratology. Coreference resolution of character mentions in screenplays becomes challenging because of the large document lengths, unique structural features like scene headers, interleaving of action and speech passages, and reliance on the accompanying video. In this work, we first adapt widely- used annotation guidelines to address domain- specific issues in screenplays. We develop an automatic screenplay parser to extract the structural information and design coreference rules based upon the structure. Our model exploits these structural features and outper- forms a benchmark coreference model on the screenplay coreference resolution task. 1 Introduction Screenplays are semi-structured text documents containing the dialogue and directions of a film. Automated screenplay analysis provides an oppor- tunity early in the creative process to offer insights into character representations and portrayals (of who interacts with whom, about what, and how), including from a diversity, inclusion, and social im- pact perspective (Ramakrishna et al., 2017; Shafaei et al., 2020; Martinez et al., 2020). A typical screen- play contains indented blocks of text that can be classified into scene headers, scene descriptions, speakers, and utterances, as shown in Figure 1 (Agarwal et al., 2014). A scene header starts a new scene and provides location and temporal informa- tion. Scene descriptions describe the characters and their actions, and the speaker and utterance blocks contain the characters’ names and speech. Screenplays can refer to a character by different names, pronouns, and nominal expressions. For Figure 1: Coreference-annotated screenplay excerpt from the movie The Shawshank Redemption (1994). Mentions of the same character are underlined with the same color. example, the screenplay excerpt shown in Figure 1 refers to the character Andy by the mentions – Andy Dufresne (name), The wife-killin’ banker (nomi- nal), and his (pronoun). Many downstream tasks need to find and map all such mentions to the cor- rect referent. For example, Gorinski and Lapata (2015) resolved pronominal mentions to their cor- rect antecedent (prior co-referring mention) to find speaker-listener and semantic relations between characters for the movie-summarization task. Chen and Choi (2016) crowdsourced character-mention labels in TV show transcripts for automatic char- acter identification. Deleris et al. (2018) used di- rect and indirect character references in utterances to study social relationships between characters. Thus, resolving character mentions in screenplays is an essential subtask in many applications. In NLP literature, this task is formally called coref- erence resolution (Jurafsky and Martin, 2009) and has been studied extensively (Sukthanker et al., 2020; Stylianou and Vlahavas, 2021). 2005 Most existing coreference datasets focus on news and web text (Pradhan et …

Content truncated to 500 words