WebCPM: Interactive Web Search for Chinese Long-form Question Answering

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8968–8988 July 9-14, 2023 ©2023 Association for Computational Linguistics WEBCPM: Interactive Web Search for Chinese Long-form Question Answering Yujia Qin1, Zihan Cai1, Dian Jin1, Lan Yan1, Shihao Liang3, Kunlun Zhu3, Yankai Lin2∗, Xu Han1, Ning Ding1, Huadong Wang1, Ruobing Xie4, Fanchao Qi1, Zhiyuan Liu1∗, Maosong Sun1∗, Jie Zhou4 1NLP Group, DCST, IAI, BNRIST, Tsinghua University, Beijing 2Gaoling School of Artificial Intelligence, Renmin University of China, Beijing 3ModelBest Inc. 4Pattern Recognition Center, WeChat AI, Tencent Inc. qyj20@mails.tsinghua.edu.cn Abstract Long-form question answering (LFQA) aims at answering complex, open-ended questions with detailed, paragraph-length responses. The de facto paradigm of LFQA necessitates two pro- cedures: information retrieval, which searches for relevant supporting facts, and information synthesis, which integrates these facts into a coherent answer. In this paper, we introduce WebCPM, the first Chinese LFQA dataset. One unique feature of WebCPM is that its infor- mation retrieval is based on interactive web search, which engages with a search engine in real time. Following WebGPT (Nakano et al., 2021), we develop a web search interface. We recruit annotators to search for relevant infor- mation using our interface and then answer questions. Meanwhile, the web search behav- iors of our annotators would be recorded. In total, we collect 5, 500 high-quality question- answer pairs, together with 15, 372 supporting facts and 125, 954 web search actions. We fine- tune pre-trained language models to imitate human behaviors for web search and to gen- erate answers based on the collected facts. Our LFQA pipeline, built on these fine-tuned mod- els, generates answers that are no worse than human-written ones in 32.5% and 47.5% of the cases on our dataset and DuReader (He et al., 2018), respectively. The interface, dataset, and codes are publicly available at https: //github.com/thunlp/WebCPM. 1 Introduction Long-form question answering (LFQA) (Fan et al., 2019) targets answering complex, open-ended ques- tions with detailed, paragraph-length responses. Current LFQA solutions generally follow the retrieve-then-synthesize paradigm, which com- prises two core ingredients: information retrieval and information synthesis. The former searches ex- ternal knowledge sources (e.g., the web) for diverse ∗Corresponding author. relevant supporting facts, and the latter integrates the collected facts into a coherent answer. One defect of the conventional LFQA paradigm is that it often resorts to non-interactive retrieval methods, which use the original question as the query to retrieve a pile of uncurated information. On the contrary, humans are able to perform inter- active web search by engaging with a search engine in real time. For a complex question, humans tend to decompose it into multiple sub-questions and ask them in sequence. By identifying and browsing relevant information, humans can improve their un- derstanding of the topic and refine their searches by asking follow-up questions or related terms. This it- erative process enables expanding the scope of their searches and improving the results they receive. Overall, interactive web search not only provides access to diverse information sources, but also re- flects …

Content truncated to 500 words