A Corpus of Gesture-Annotated Dialogues for Monologue-to-Dialogue Generation from Personal Narratives

A Corpus of Gesture-Annotated Dialogues for Monologue-to-Dialogue Generation from Personal Narratives Zhichao Hu1, Michelle Dick1, Chung-Ning Chang1, Kevin Bowden1 Michael Neff2, Jean E. Fox Tree1, Marilyn Walker1 University of California, Santa Cruz1; University of California, Davis2 1156 High Street, Santa Cruz, CA, USA; 1 Shields Ave, Davis, CA, USA {zhu, mdick, cchan60, kkbowden}@ucsc.edu, mpneff@ucdavis.edu, {foxtree, mawalker}@ucsc.edu Abstract Story-telling is a fundamental and prevalent aspect of human social behavior. In the wild, stories are told conversationally in social settings, often as a dialogue and with accompanying gestures and other nonverbal behavior. This paper presents a new corpus, the STORY DIALOGUE WITH GESTURES (SDG) corpus, consisting of 50 personal narratives regenerated as dialogues, complete with annotations of gesture placement and accompanying gesture forms. The corpus includes dialogues generated by human annotators, gesture annotations on the human generated dialogues, videos of story dialogues generated from this representation, video clips of each gesture used in the gesture annotations, and annotations of the original personal narratives with a deep representation of story called a STORY INTENTION GRAPH. Our long term goal is the automatic generation of story co-tellings as animated dialogues from the STORY INTENTION GRAPH. We expect this corpus to be a useful resource for researchers interested in natural language generation, intelligent virtual agents, generation of nonverbal behavior, and story and narrative representations. Keywords: storytelling, personal narrative, dialogue, virtual agents, gesture generation, personality 1. Introduction Sharing experiences by story-telling is a fundamental and prevalent aspect of human social behavior (Bruner, 1991; Bohanek et al., 2006; Labov and Waletzky, 1997; Nelson, 2003). In the wild, stories are told conversationally in social settings, often as a dialogue and with accompanying gestures and other nonverbal behavior (Tolins et al., 2016a). Story- telling in the wild serves many different social functions: e.g. stories are used to persuade, share troubles, establish shared values, learn social behaviors, and entertain (Ryokai et al., 2003; Pennebaker and Seagal, 1999; Gratch et al., 2013). Previous research has also shown that conveying informa- tion in the form of a dialogue is more engaging, effective and persuasive compared to a monologue (Lee et al., 1998; Craig et al., 2000; Suzuki and Yamada, 2004; Andr´e et al., 2000). Thus our long term goal is the automatic generation of story co-tellings as animated dialogues. Given a deep representation of the story, many different versions of the story can be generated, both dialogic and monologic (Lukin and Walker, 2015; Lukin et al., 2015). This paper presents a new corpus, the STORY DIALOGUE WITH GESTURES (SDG) corpus, consisting of 50 personal narratives regenerated as dialogues by human annotators, complete with annotations of gesture placement and accom- panying gesture forms. These annotations can be supple- mented programmatically to produce changes in the nonver- bal dialogic behavior in gesture rate, expanse and speed. We have thus used these to generate tellings that vary the per- sonality of the teller (introverted vs. extroverted) (Hu et al., 2015). An example original monologic personal narrative about having cats as pets is shown in Figure …

Content truncated to 500 words