What Should I Ask? Using Conversationally Informative Rewards for Goal-oriented Visual Dialog.

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6442–6451 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 6442 What Should I Ask? Using Conversationally Informative Rewards for Goal-Oriented Visual Dialogue Pushkar Shuklar1, Carlos Elmadjian2, Richika Sharan1, Vivek Kulkarni3, William Yang Wang1, Matthew Turk1 1University of California, Santa Barbara 2University of S˜ao Paulo 3Stanford University 1{pushkarshukla,richikasharan,wangwilliamyang,mturk}@ucsb.edu 2 elmad@ime.usp.br 3 viveksck@stanford.edu Abstract The ability to engage in goal-oriented conver- sations has allowed humans to gain knowl- edge, reduce uncertainty, and perform tasks more efficiently. Artificial agents, however, are still far behind humans in having goal- driven conversations. In this work, we fo- cus on the task of goal-oriented visual dia- logue, aiming to automatically generate a se- ries of questions about an image with a sin- gle objective. This task is challenging, since these questions must not only be consistent with a strategy to achieve a goal, but also consider the contextual information in the im- age. We propose an end-to-end goal-oriented visual dialogue system, that combines rein- forcement learning with regularized informa- tion gain. Unlike previous approaches that have been proposed for the task, our work is motivated by the Rational Speech Act frame- work, which models the process of human in- quiry to reach a goal. We test the two versions of our model on the GuessWhat?! dataset, ob- taining significant results that outperform the current state-of-the-art models in the task of generating questions to find an undisclosed ob- ject in an image. 1 Introduction Building natural language models that are able to converse towards a specific goal is an active area of research that has attracted a lot of attention in recent years. These models are vital for efficient human-machine collaboration, such as when inter- acting with personal assistants. In this paper, we focus on the task of goal-oriented visual dialogue, which requires an agent to engage in conversations about an image with a predefined objective. The task presents some unique challenges. Firstly, the conversations should be consistent with the goals of the agent. Secondly, the conversations between two agents must be coherent with the common vi- Figure 1: An example of goal-oriented visual dialogue for finding an undisclosed object in an image through a series of questions. On the left, we ask a human to guess the unknown object in the image. On the right, we use the baseline model proposed by Strub et al. (Strub et al., 2017). While the human is able to nar- row down the search space relatively faster, the artifi- cial agent is not able to adopt a clear strategy for guess- ing the object. sual feedback. Finally, the agents should come up with a strategy to achieve the objective in the shortest possible way. This is different from a nor- mal dialogue system where there is no constraint on the length of a conversation. Inspired by the success of Deep Reinforcement Learning, many recent works have also used it for building models …

Content truncated to 500 words