VoiSeR: A New Benchmark for Voice-Based Search Refinement

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2321–2329 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2321 VoiSeR: A New Benchmark for Voice-Based Search Refinement Simone Filice, Giuseppe Castellucci, Marcus Collins, Eugene Agichtein, Oleg Rokhlenko Amazon Seattle, USA {filicesf,giusecas,collmr,eugeneag,olegro}@amazon.com Abstract Voice assistants, e.g., Alexa or Google As- sistant, have dramatically improved in recent years. Supporting voice-based search, explo- ration, and refinement are fundamental tasks for voice assistants, and remain an open chal- lenge. For example, when using voice to search an online shopping site, a user often needs to refine their search by some aspect or facet. This common user intent is usually available through a “filter-by” interface on on- line shopping websites, but is challenging to support naturally via voice, as the intent of re- finements must be interpreted in the context of the original search, the initial results, and the available product catalogue facets. To our knowledge, no benchmark dataset exists for training or validating such contextual search understanding models. To bridge this gap, we introduce the first large-scale dataset of voice- based search refinements, VoiSeR, consisting of about 10,000 search refinement utterances, collected using a novel crowdsourcing task. These utterances are intended to refine a pre- vious search, with respect to a search facet or attribute (e.g., brand, color, review rating, etc.), and are manually annotated with the spe- cific intent. This paper reports qualitative and empirical insights into the most common and challenging types of refinements that a voice- based conversational search system must sup- port. As we show, VoiSeR can support re- search in conversational query understanding, contextual user intent prediction, and other conversational search topics to facilitate the de- velopment of conversational search systems. 1 Introduction Modern voice assistants, such as Amazon Alexa or Apple Siri, make use of Natural Language Un- derstanding (NLU) techniques to perform several tasks. Some of the most popular functions offered by these systems are based on voice-search: mil- lions use voice assistants to access information or search for music, products or local restaurants and stores. However, search experience with a voice assistant remains limited. The current generation of these systems mostly supports single-turn inter- actions, and does not naturally support more com- plex search needs, which often require refinements to narrow, broaden or change the initial search. Supporting refinement is a fundamental aspect of search systems, and it is done in a variety of ways in Web-based user interfaces, e.g., through query suggestion or explicit facets navigation or filter- ing. For example, in an e-Commerce search, a user may want to refine their search with respect to some facet or attribute (e.g., brand or price); this critical functionality is supported on most e- Commerce websites. However, this kind of interac- tion is challenging to support via voice-based dia- logue interfaces, as interpreting such refinements requires modeling the original search intent, the initial results, and the available result facets. To the best of our knowledge, no large scale …

Content truncated to 500 words