“It doesn’t look good for a date”: Transforming Critiques into Preferences for Conversational Recommendation Systems

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1913–1918 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1913 "It doesn’t look good for a date": Transforming Critiques into Preferences for Conversational Recommendation Systems Victor S. Bursztyn1, Jennifer Healey2, Nedim Lipka2, Eunyee Koh2, Doug Downey1,3, and Larry Birnbaum1 1Department of Computer Science, Northwestern University, Evanston, IL, USA 2Adobe Research, San Jose, CA, USA 3Allen Institute for Artificial Intelligence, Seattle, WA, USA v-bursztyn@u.northwestern.edu, {jehealey,lipka,eunyee}@adobe.com {d-downey,l-birnbaum}@northwestern.edu Abstract Conversations aimed at determining good rec- ommendations are iterative in nature. People often express their preferences in terms of a critique of the current recommendation (e.g., “It doesn’t look good for a date”), requiring some degree of common sense for a prefer- ence to be inferred. In this work, we present a method for transforming a user critique into a positive preference (e.g., “I prefer more roman- tic”) in order to retrieve reviews pertaining to potentially better recommendations (e.g., “Per- fect for a romantic dinner”). We leverage a large neural language model (LM) in a few- shot setting to perform critique-to-preference transformation, and we test two methods for retrieving recommendations: one that matches embeddings, and another that fine-tunes an LM for the task. We instantiate this approach in the restaurant domain and evaluate it using a new dataset of restaurant critiques. In an ablation study, we show that utilizing critique- to-preference transformation improves recom- mendations, and that there are at least three general cases that explain this improved per- formance. 1 Introduction Conversational recommendation systems (CRSs) are dialog-based systems that aim to refine a set of options over multiple turns of a conversation, en- visioning more natural interactions and better user modeling than in non-conversational approaches. However, the resulting dialogs still do not neces- sarily reflect how real conversations unfold. Most CRSs fall into two categories: they either frame the problem as a slot-filling task within a prede- fined feature space, such as Sun and Zhang (2018); Zhang et al. (2018); Budzianowski et al. (2018), which is closer to how people make decisions but not as flexible as real conversations; or they elicit preferences by asking users to rate specific items, such as Christakopoulou et al. (2016), which is in- dependent of a feature space but not as natural to users. Figure 1: An example of our system transforming a critique into a positive preference and then using a cus- tomer testimonial to sell the user on a new option. When we examine situations involving real hu- man agents (Lyu et al., 2021), decisions typically require multiple rounds of recommendations by the agent and critiques by the user, with the agent con- tinuously improving the recommendations based upon user preferences that can be inferred from such critiques. These inferences can be compared to the types of common sense inferences that have been studied recently with LMs (Davison et al., 2019; Majumder et al., 2020; Jiang et al., 2021). However, use of LMs for critique interpretation remains underex- plored, despite the important …

Content truncated to 500 words