Building Task-Oriented Visual Dialog Systems Through Alternative Optimization Between Dialog Policy and Language Generation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 143–153, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 143 Building Task-Oriented Visual Dialog Systems Through Alternative Optimization Between Dialog Policy and Language Generation Mingyang Zhou Josh Arnold Department of Computer ScienceZhou Yu University of California, Davis {minzhou, jarnold, joyu}@ucdavis.edu Abstract Reinforcement learning (RL) is an effective approach to learn an optimal dialog pol- icy for task-oriented visual dialog systems. A common practice is to apply RL on a neural sequence-to-sequence (seq2seq) frame- work with the action space being the out- put vocabulary in the decoder. However, it is difficult to design a reward function that can achieve a balance between learning an ef- fective policy and generating a natural dia- log response. This paper proposes a novel framework that alternatively trains a RL policy for image guessing and a supervised seq2seq model to improve dialog generation quality. We evaluate our framework on the Guess- Which task and the framework achieves the state-of-the-art performance in both task com- pletion and dialog quality. 1 Introduction Visually-grounded conversational artificial intel- ligence (AI) is an important field that explores the extent intelligent systems are able to hold meaningful conversations regarding visual con- tent. Visually-grounded conversational AI can be applied to a wide range of real-world tasks, includ- ing assisting blind people to navigate their sur- roundings, online recommendation systems, and analysing mass amounts of visual media through natural language. Current approaches to these tasks involve an end-to-end framework that maps the multi-modal context to a deep vector and in order to decode a natural dialog response. This framework can be trained through supervised learning (SL) with the objective to maximize the distribution of the response given a human-human dialog history. Given a large conversational data, the neural end-to-end system can effectively learn to generate coherent and natural language. While much success has been achieved by ap- plying neural sequence to sequence models to open visual grounding conversation, the visual di- alog system also needs to learn an optimal strat- egy to efficiently accomplish an external goal through natural conversations. To address this is- sue, various image guessing tasks such us Guess- Which (Chattopadhyay et al., 2017) and Guess- What (de Vries et al., 2016) are proposed to eval- uate a visual-grounded conversational agent on its ability to retrieve visual content via conversing in natural language. To obtain an optimal dialog pol- icy, reinforcement learning (RL) is introduced to enable the neural end-to-end framework to model a more effective action distribution by exploring different dialog strategies. With the application of RL, the visual dialog system can generate more consistent responses and achieve a higher level of engagement in the conversation when compared to a dialog system trained via SL (Das et al., 2017b; Zhang et al., 2017). A typical way to apply RL on a dialog system is to assign a task-related re- ward to influence the utterance …

Content truncated to 500 words