TVShowGuess: Character Comprehension in Stories as Speaker Guessing

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4267 - 4287 July 10-15, 2022 ©2022 Association for Computational Linguistics TVSHOWGUESS: Character Comprehension in Stories as Speaker Guessing Yisi Sang1∗ Xiangyang Mou2∗ Mo Yu3∗ Shunyu Yao4 Jing Li5 Jeffrey Stanton1 1Syracuse University 2Rensselaer Polytechnic Institute 3Pattern Recognition Center, WeChat AI 4Princeton University 5New Jersey Institute of Technology yisang@syr.edu moux4@rpi.edu moyumyu@tencent.com Abstract We propose a new task for assessing machines’ ability to understand fictional characters in narrative stories. The task, TVSHOWGUESS, builds on the scripts of TV series and takes the form of guessing the anonymous main charac- ters based on the backgrounds of the scenes and dialogues. Our human study supports that this form of task covers comprehension of mul- tiple types of character persona, including un- derstanding characters’ personalities, facts and memories of personal experience, which are well aligned with the psychological and liter- ary theories about the theory of mind (ToM) of human beings on understanding fictional char- acters during reading. We further propose new model architectures to support the contextu- alized encoding of long scene texts. Experi- ments show that our proposed approaches sig- nificantly outperform baselines, yet still largely lag behind the (nearly perfect) human perfor- mance. Our work serves as a first step toward the goal of narrative character comprehension.1 1 Introduction Stories have two essential elements, plots and characters (McKee, 1997). Character comprehen- sion has been widely recognized as key to un- derstanding stories in psychological, literary and educational research (Bower and Morrow, 1990; Kennedy et al., 2013; Currie, 2009; Paris and Paris, 2003). When reading stories, humans build men- tal models for characters based on their personae, which helps people to explain a character’s emo- tional status (Gernsbacher et al., 1998), identity, un- derstand future behaviors (Mead, 1990), and even make counterfactual inferences for stories about that character (Fiske et al., 1979). The ultimate goal of creating a character com- prehension system is to equip a machine with prac- ∗Authors contributed equally to this paper. Mo Yu is the corresponding author. 1Our code and data are released at https://github. com/YisiSang/TVSHOWGUESS. Figure 1: A scene example from TVSHOWGUESS. The character Amy can be determined within the scene or with the fact of her relationship; while guessing Sheldon would require memory of the character from previous episodes. tical capabilities that emulate what humans can accomplish. For example, understanding personae can facilitate story memorization and generation of new statements consistent with the story (Riedl and Young, 2010). Such capabilities could be valuable in the construction of dialog engines that help peo- ple address practical problems such as those expe- rienced in customer service encounters (Mairesse and Walker, 2007; Zhang et al., 2018; Urbanek et al., 2019). More importantly, understanding the persona of a particular person can help chatbots to understand the intention behind a human user’s lan- guage (Bender and Koller, 2020), which can lead to better services and ultimately give systems the ability to …

Content truncated to 500 words