Integrating Deep Event-Level and Script-Level Information for Script Event Prediction

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1004–1015 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1004 Contextualized Query Embeddings for Conversational Search Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo Abstract This paper describes a compact and effective model for low-latency passage retrieval in con- versational search based on learned dense rep- resentations. Prior to our work, the state-of- the-art approach uses a multi-stage pipeline comprising conversational query reformula- tion and information retrieval modules. De- spite its effectiveness, such a pipeline often includes multiple neural models that require long inference times. In addition, indepen- dently optimizing each module ignores de- pendencies among them. To address these shortcomings, we propose to integrate conver- sational query reformulation directly into a dense retrieval model. To aid in this goal, we create a dataset with pseudo-relevance labels for conversational search to overcome the lack of training data and to explore different train- ing strategies. We demonstrate that our model effectively rewrites conversational queries as dense representations in conversational search and open-domain question answering datasets. Finally, after observing that our model learns to adjust the L2 norm of query token embed- dings, we leverage this property for hybrid re- trieval and to support error analysis. 1 Introduction With the growing popularity of virtual assistants (e.g., Alexa and Siri), information seeking through dialogues has attracted many researchers’ atten- tion. To facilitate research on conversational search (ConvS), Dalton et al. (2019) organized the TREC Conversational Assistance Track (CAsT) and de- fined ConvS as the task of iteratively retrieving passages in response to user queries in a conversa- tion session. An example conversation in the CAsT dataset is shown at the top of Figure 1(a). There are two main challenges for the task of conversational search: (1) User utterances are of- ten ambiguous when treated as stand-alone queries since omission, coreference, and other related lin- guistic phenomena are common in natural human (a) Multi-stage pipeline for conversational search q<4; q4 Conversational Query Reformulator BM25 BERT Re-ranker Passage Encoder Conversational Query Encoder Eq<4;q4 Ep Corpus Pre-compute What are lung cancer's symptoms? Relevant passage Gradient (b) End-to-end conversational search What is throat cancer? Is it treatable? Tell me about lung cancer. What are its symptoms? q1 q2 q3 q4 CAsT Example q<4; q4 (a) Multi-stage pipeline for conversational search: fine-tuned language models are used to reformulate user utterances for the downstream IR pipeline. (a) Multi-stage pipeline for conversational search q<4; q4 Conversational Query Reformulator BM25 BERT Re-ranker Passage Encoder Conversational Query Encoder Eq<4;q4 Ep Corpus Pre-compute What are lung cancer's symptoms? Relevant passage Gradient (b) End-to-end conversational search What is throat cancer? Is it treatable? Tell me about lung cancer. What are its symptoms? q1 q2 q3 q4 CAsT Example q<4; q4 (b) End-to-end conversational search: query reformulation is directly incorporated into the IR pipeline, thus enabling end-to-end training. Figure 1: A comparison between (a) a multi-stage pipeline and (b) our proposed method for …

Content truncated to 500 words