Recommendation as a Communication Game: Self-Supervised Bot-Play for Goal-oriented Dialogue

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1951–1961, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1951 Recommendation as a Communication Game: Self-Supervised Bot-Play for Goal-oriented Dialogue Dongyeop Kang♥ Anusha Balakrishnan♣ Pararth Shah♣ Paul Crook♣ Y-Lan Boureau♣ Jason Weston♣ ♥Carnegie Mellon University, ♣Facebook AI dongyeok@cs.cmu.edu anusha.bala28@gmail.com {pararths,pacrook,ylan,jase}@fb.com Abstract Traditional recommendation systems produce static rather than interactive recommendations invariant to a user’s specific requests, clarifi- cations, or current mood, and can suffer from the cold-start problem if their tastes are un- known. These issues can be alleviated by treating recommendation as an interactive di- alogue task instead, where an expert recom- mender can sequentially ask about someone’s preferences, react to their requests, and recom- mend more appropriate items. In this work, we collect a goal-driven recommendation di- alogue dataset (GoRecDial), which consists of 9,125 dialogue games and 81,260 conver- sation turns between pairs of human workers recommending movies to each other. The task is specifically designed as a cooperative game between two players working towards a quan- tifiable common goal. We leverage the dataset to develop an end-to-end dialogue system that can simultaneously converse and recommend. Models are first trained to imitate the behavior of human players without considering the task goal itself (supervised training). We then fine- tune our models on simulated bot-bot conver- sations between two paired pre-trained mod- els (bot-play), in order to achieve the dialogue goal. Our experiments show that models fine- tuned with bot-play learn improved dialogue strategies, reach the dialogue goal more often when paired with a human, and are rated as more consistent by humans compared to mod- els trained without bot-play. The dataset and code are publicly available through the ParlAI framework1. 1 Introduction Traditional recommendation systems factorize users’ historical data (i.e., ratings on movies) to extract common preference patterns (Koren et al., 1https://github.com/facebookresearch/ParlAI 2009; He et al., 2017b). However, besides mak- ing it difficult to accommodate new users because of the cold-start problem, relying on aggregated history makes these systems static, and prevents users from making specific requests, or exploring a temporary interest. For example, a user who usu- ally likes horror movies, but is in the mood for a fantasy movie, has no way to indicate their prefer- ence to the system, and would likely get a recom- mendation that is not useful. Further, they cannot iterate upon initial recommendations with clarifi- cations or modified requests, all of which are best specified in natural language. Recommending through dialogue interactions (Reschke et al., 2013; W¨arnestål, 2005) offers a promising solution to these problems, and recent work by Li et al. (2018) explores this approach in detail. However, the dataset introduced in that work does not capture higher-level strategic be- haviors that can impact the quality of the rec- ommendation made (for example, it may be bet- ter to elicit user preferences first, before making a recommendation). This makes it difficult for models trained …

Content truncated to 500 words