Database Search Results Disambiguation for Task-Oriented Dialog Systems

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1158 - 1173 July 10-15, 2022 ©2022 Association for Computational Linguistics Database Search Results Disambiguation for Task-Oriented Dialog Systems Kun Qian†§, Satwik Kottur‡, Ahmad Beirami‡¶, Shahin Shayandeh‡, Paul Crook‡, Alborz Geramifard‡, Zhou Yu†, Chinnadhurai Sankar‡ †Columbia University ‡Meta AI {kq2157, zy2461}@columbia.edu, beirami@google.com {skottur, shn, pacrook, alborzg, chinnadhurai}@fb.com Abstract As task-oriented dialog systems are becoming increasingly popular in our lives, more real- istic tasks have been proposed and explored. However, new practical challenges arise. For instance, current dialog systems cannot ef- fectively handle multiple search results when querying a database, due to the lack of such scenarios in existing public datasets. In this pa- per, we propose Database Search Result (DSR) Disambiguation, a novel task that focuses on disambiguating database search results, which enhances user experience by allowing them to choose from multiple options instead of just one. To study this task, we augment the pop- ular task-oriented dialog datasets (MultiWOZ and SGD) with turns that resolve ambiguities by (a) synthetically generating turns through a pre-defined grammar, and (b) collecting human paraphrases for a subset. We find that train- ing on our augmented dialog data improves the model’s ability to deal with ambiguous scenar- ios, without sacrificing performance on unmod- ified turns. Furthermore, pre-fine tuning and multi-task learning help our model to improve performance on DSR-disambiguation even in the absence of in-domain data, suggesting that it can be learned as a universal dialog skill. Our data and code will be made publicly available. 1 Introduction Task-oriented dialog systems have been widely de- ployed for popular virtual assistants, like Siri and Google Assistant. They help people with tasks such as booking restaurants and looking for a hotel by searching databases with constraints provided by users. After retrieving a result from the database, a system may continue by conducting actions like making a reservation or providing more informa- tion about receiving the result. However, there can be multiple results from the database that match § Work done during KQ‘s internship at Meta AI. ¶ Currently at Google Research. Figure 1: Examples of disambiguation turns over three different domains. the same constraints. For example, as shown in Fig. 1, the system finds two available hotels at dif- ferent locations when the user is asking the system to help book a hotel. This kind of ambiguity stops system from proceeding until the system finds out which result the user looks for. Therefore, we need to enhance the system with the ability to resolve such ambiguity brought out by multiple items re- turned from database search. We call this type of ambiguity as database search result ambiguity (DSR-ambiguity). Different from semantic ambiguous words (e.g. “orange” can be referred as either color or fruit), the DSR-ambiguity focuses on results from mul- tiple database search results. Solving such disam- biguation tasks consists of two steps: asking clar- ification questions and understanding user’s cor- responding answers. While there is …

Content truncated to 500 words