Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7135 - 7146 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering Jiawei Zhou1, Xiaoguang Li2, Lifeng Shang2, Lan Luo3, Ke Zhan3, Enrui Hu3, Xinyu Zhang3, Hao Jiang3, Zhao Cao3, Fan Yu3, Xin Jiang2, Qun Liu2, Lei Chen1 1The Hong Kong University of Science and Technology 2Huawei Noah’s Ark Lab 3Distributed and Parallel Software Lab, Huawei 1{jzhoubu,leichen}@ust.hk 2,3{lixiaoguang11,Shang.Lifeng,luolan13,zhanke2,huenrui1,zhangxinyu35 jianghao66,caozhao1,fan.yu,Jiang.Xin,qun.liu}@huawei.com Abstract To alleviate the data scarcity problem in train- ing question answering systems, recent works propose additional intermediate pre-training for dense passage retrieval (DPR). However, there still remains a large discrepancy between the provided upstream signals and the downstream question-passage relevance, which leads to less improvement. To bridge this gap, we propose the HyperLink-induced Pre-training (HLP), a method to pre-train the dense retriever with the text relevance induced by hyperlink-based topology within Web documents. We demon- strate that the hyperlink-based structures of dual-link and co-mention can provide effective relevance signals for large-scale pre-training that better facilitate downstream passage re- trieval. We investigate the effectiveness of our approach across a wide range of open-domain QA datasets under zero-shot, few-shot, multi- hop, and out-of-domain scenarios. The exper- iments show our HLP outperforms the BM25 by up to 7 points as well as other pre-training methods by more than 10 points in terms of top-20 retrieval accuracy under the zero-shot scenario. Furthermore, HLP significantly out- performs other pre-training methods under the other scenarios. 1 Introduction Open-domain question answering (OpenQA) aims to answer factual open questions with a large ex- ternal corpus of passages. Current approaches to OpenQA usually adopt a two-stage retriever-reader paradigm (Chen et al., 2017; Zhu et al., 2021) to fetch the final answer span. The performance of OpenQA systems is largely bounded by the re- triever as it determines the evidential documents for the reader to examine. Traditional retrievers, such as TF-IDF and BM25 (Robertson and Zaragoza, 2009), are considered incapable of adapting to sce- Our code and trained models are available at https: //github.com/jzhoubu/HLP. In 2011 he directed his first international feature film, romantic comedy "Letters to Santa (Listy do M.)" Who directs the romantic comedy "Letters to Santa"? Letters to Santa (Polish: Listy do M.), alternatively known as Letters to St. Nicholas, is a 2011 Polish-language romantic comedy film, directed by the director Mitja Okorn. The action takes place during one single Christmas Eve, when a few adults find the loves of their lives. The film's plot refers to the 2003 romantic comedy "Love Actually", though events of the movie differ from the ones in the 2003 film. It was shot in Warsaw, Poland from 27 January to March 2011. The movie had 3 sequels, Letters to Santa 2 released in 2015, Letters to Santa 3 in 2017, and Listy do M. 4 that was produced in 2020 and rescheduled to premiere after Mitja Okorn Letters to Santa (film) Human Query Our HLP Query …

Content truncated to 500 words