Hierarchical Transformer for Task Oriented Dialog Systems

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 520–534 June 6–11, 2021. ©2021 Association for Computational Linguistics 520 Open-Domain Question Answering Goes Conversational via Question Rewriting Raviteja Anantha⭑♦, Svitlana Vakulenko⭑♠♥, Zhucheng Tu♦, Shayne Longpre♦, Stephen Pulman♦, Srinivas Chappidi♦ ♦Apple Inc. ♠University of Amsterdam, Amsterdam, the Netherlands Abstract We introduce a new dataset for Question Rewriting in Conversational Context (QReCC), which contains 14K conversations with 80K question-answer pairs. The task in QReCC is to find answers to conversational questions within a collection of 10M web pages (split into 54M passages). Answers to questions in the same conversation may be distributed across several web pages. QReCC provides annotations that allow us to train and evaluate individual subtasks of question rewriting, passage retrieval and reading comprehension required for the end-to-end conversational question answering (QA) task. We report the effectiveness of a strong baseline approach that combines the state-of-the-art model for question rewriting, and competitive models for open-domain QA. Our results set the first baseline for the QReCC dataset with F1 of 19.10, compared to the human upper bound of 75.45, indicating the difficulty of the setup and a large room for improvement. 1 Introduction It is often not possible to address a complex infor- mation need with a single question. Consequently, there is a clear need to extend open-domain ques- tion answering (QA) to a conversational setting. This task is commonly referred to as conversational (interactive or sequential) QA (Webb, 2006; Saeidi et al., 2018; Reddy et al., 2019). Conversational QA requests an answer conditioned on both the question and the previous conversation turns as context. Previously proposed large-scale bench- marks for conversational QA, such as QuAC and CoQA, limit the topic of conversation to the content of a single document. In practice, however, the an- swers can be distributed across several documents ⭑ Equal contribution. ♥ Work done as an intern at Apple Inc. Figure 1: A snippet of a sample conversation from QReCC with question rewrites and answer provenance links. Orange indicates coreference cases where the highlighted token should be replaced with its an- tecedent (in bold). Blue indicates the tokens that should be generated to make the question unambiguous outside of the conversational context. that are relevant to the conversation, or the topic of the conversation may also drift. To investigate this phenomena and develop approaches suitable for the complexities of this task, we introduce a new dataset for open-domain conversational QA, called QReCC.1 The dataset consists of 13.6K conversa- tions with an average of 6 turns per conversation. A conversation in QReCC consists of a sequence of question-answer pairs. The answers to questions were produced by human annotators, who looked up relevant information on the web using a search engine. QReCC is therefore the first large-scale dataset for conversational QA that incorporates an information retrieval subtask. QReCC is accompa- nied with scripts for building a collection of pas- sages from the Common Crawl and the Wayback Machine …

Content truncated to 500 words