A Dynamic Speaker Model for Conversational Interactions

Proceedings of NAACL-HLT 2019, pages 2772–2785 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2772 A Dynamic Speaker Model for Conversational Interactions Hao Cheng Hao Fang Mari Ostendorf University of Washington {chenghao,hfang,ostendorf}@uw.edu Abstract Individual differences in speakers are reflected in their language use as well as in their inter- ests and opinions. Characterizing these differ- ences can be useful in human-computer inter- action, as well as analysis of human-human conversations. In this work, we introduce a neural model for learning a dynamically up- dated speaker embedding in a conversational context. Initial model training is unsuper- vised, using context-sensitive language gen- eration as an objective, with the context be- ing the conversation history. Further fine- tuning can leverage task-dependent supervised training. The learned neural representation of speakers is shown to be useful for content ranking in a socialbot and dialog act prediction in human-human conversations.1 1 Introduction Representing language in context is key to improv- ing natural language processing (NLP). There are a variety of useful contexts, including word his- tory, related documents, author/speaker informa- tion, social context, knowledge graphs, visual or situational grounding, etc. This paper addresses the problem of modeling the speaker. Account- ing for author/speaker variations has been shown to be useful in many NLP tasks, including lan- guage understanding (Hovy and Søgaard, 2015; Volkova et al., 2013), language generation (Mirkin et al., 2015; Li et al., 2016), human-computer di- alog policy (Bowden et al., 2018), query comple- tion (Jaech and Ostendorf, 2018; Shokouhi, 2013), comment recommendation (Agarwal et al., 2011) and more. In this work, we specifically focus on dialogs, including both human-computer (social- bot) and human-human conversations. 1The implementation of code is available at https://github.com/hao-cheng/dynamic_ speaker_model.git While many studies rely only on discrete meta- data and/or demographic information, such infor- mation is not always available. Thus, it is of in- terest to learn about the speaker from the language directly, as it relates to the person’s interests and speaking style. Motivated by the success of un- supervised contextualized representation learning for words and documents (Mikolov et al., 2013; Kiros et al., 2015; McCann et al., 2017; Peters et al., 2018; Devlin et al., 2019), our approach is to use unsupervised learning with a neural model of a speaker’s dialog history. The model uses latent speaker mode vectors for representing a speaker turn as in (Cheng et al., 2017), which provides a framework for analysis of what the model learns about speaking style. Further, the model is struc- tured to allow a dynamic update of the speaker vector at each turn in a dialog, in order to capture changes over time and improve the speaker repre- sentation with added data. The speaker embeddings can be used as context in conversational language understanding tasks, e.g., as an additional input in dialog policy predic- tion in human-computer dialogs or in understand- ing dialog acts in human-human dialogs. In the su- pervised training of such tasks, the speaker model can be fine-tuned. This work makes …

Content truncated to 500 words