Self-Attentional Models Application in Task-Oriented Dialogue Generation Systems

Proceedings of Recent Advances in Natural Language Processing, pages 1031–1040, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_119 1031 Self-Attentional Models Application in Task-Oriented Dialogue Generation Systems Mansour Saffar Mehrjardi, Amine Trablesi, Osmar R. Za¨ıane Department of Computing Science, University of Alberta {saffarme,atrabels,zaiane}@ualberta.ca Abstract Self-attentional models are a new paradigm for sequence modelling tasks which differ from common sequence modelling methods, such as recurrence- based and convolution-based sequence learning, in the way that their architecture is only based on the attention mechanism. Self-attentional models have been used in the creation of the state-of-the-art models in many NLP tasks such as neural machine translation, but their usage has not been explored for the task of training end-to- end task-oriented dialogue generation systems yet. In this study, we apply these models on the three different datasets for training task-oriented chatbots. Our finding shows that self-attentional models can be exploited to create end-to-end task- oriented chatbots which not only achieve higher evaluation scores compared to recurrence-based models, but also do so more efficiently. 1 Introduction Task-oriented chatbots are a type of dialogue gen- eration system which tries to help the users accom- plish specific tasks, such as booking a restaurant table or buying movie tickets, in a continuous and uninterrupted conversational interface and usually in as few steps as possible. The development of such systems falls into the Conversational AI do- main which is the science of developing agents which are able to communicate with humans in a natural way (Ram et al., 2018). Digital assistants such as Apple’s Siri, Google Assistant, Amazon Alexa, and Alibaba’s AliMe are examples of suc- cessful chatbots developed by giant companies to engage with their customers. There are mainly two different ways to cre- ate a task-oriented chatbot which are either using set of hand-crafted and carefully-designed rules or use corpus-based method in which the chatbot can be trained with a relatively large corpus of conversational data. Given the abundance of di- alogue data, the latter method seems to be a bet- ter and a more general approach for developing task-oriented chatbots. The corpus-based method also falls into two main chatbot design architec- tures which are pipelined and end-to-end archi- tectures (Chen et al., 2017). End-to-end chatbots are usually neural networks based (Shang et al., 2015; Dodge et al., 2015; Wen et al., 2016; Eric and Manning, 2017a) and thus can be adapted to new domains by training on relevant dialogue datasets for that specific domain. Furthermore, all sequence modelling methods can also be used in training end-to-end task-oriented chatbots. A se- quence modelling method receives a sequence as input and predicts another sequence as output. For example in the case of machine translation the in- put could be a sequence of words in a given lan- guage and the output would be a sentence in a sec- ond language. In a dialogue system, an utterance is the input and the predicted sequence of words would be the corresponding response. Self-attentional models are a new paradigm for sequence modelling …

Content truncated to 500 words