On the Vector Representation of Utterances in Dialogue Context

On the Vector Representation of Utterances in Dialogue Context Louisa Pragst, Niklas Rach, Wolfgang Minker, Stefan Ultes Ulm University, Ulm University, Ulm University, Cambridge University Ulm, Germany, Ulm, Germany, Ulm, Germany, Cambridge, UK louisa.pragst@uni-ulm.de, niklas.rach@uni-ulm.de,wolfgang.minker@uni-ulm.de, su259@cam.ac.uk Abstract In recent years, the representation of words as vectors in a vector space, also known as word embeddings, has achieved a high degree of attention in the research community and the benefits of such a representation can be seen in the numerous applications that utilise it. In this work, we introduce dialogue vector models, a new language resource that represents dialogue utterances in vector space and captures the semantic meaning of those utterances in the dialogue context. We examine how the word vector approach can be applied to utterances in a dialogue to generate a meaningful representation of them in vector space. Utilising existing dialogue corpora and word vector models, we create dialogue vector models and show that they capture relevant semantic information by comparing them to manually annotated dialogue acts. Furthermore, we discuss potential areas of application for dialogue vector models, such as dialogue act annotation, learning of dialogue strategies, intent detection and paraphrasing. Keywords: sentence embedding, sentence representation, dialogue embedding, dialogue representation, dialogue act detection 1. Introduction The representation of words in vector space (Mikolov et al., 2013a), also known as word2vec, has been widely success- ful in the research community. It has been utilised in a num- ber of research areas, such as machine translation (Mikolov et al., 2013b), sentiment classification (Xue et al., 2014; Zhang et al., 2015), named entity recognition (Sienˇcnik, 2015) and document classification (Kusner et al., 2015), among many others. Considering the success and flexibility with which word vector models can be employed in numer- ous applications, we aspire to investigate whether applying the word2vec approach to dialogue utterances yields simi- lar results. In this paper, we introduce a new language resource: di- alogue vector models (DVMs), a representation of utter- ances in vector space that takes into account dialogue con- text. We investigate what adaptations are needed to the word2vec approach in order to generate such models and evaluate whether they capture information that is beneficial in a dialogue context. Furthermore, we highlight several promising areas of application that might benefit from the use of DVMs. In the following, we discuss related work in Section 2, be- fore introducing our approach to the generation of DVMs in Section 3. Here, we also evaluate the validity of the gen- erated DVMs. In Section 4, we propose potential applica- tions for those models. Finally, we draw our conclusion in Section 5. 2. Related Work A number of different approaches to the representation of sentences in vector space have been proposed, e.g. utilising recurrent neural networks (Sutskever et al., 2014; Palangi et al., 2016), convolutional neural networks (Shen et al., 2014; Kalchbrenner et al., 2014; Hu et al., 2014) and au- toencoders (Socher et al., 2011). Those approaches typ- ically do not take into account surrounding sentences for …

Content truncated to 500 words