DuReadervis: A Chinese Dataset for Open-domain Document Visual Question Answering

Findings of the Association for Computational Linguistics: ACL 2022, pages 1338 - 1351 May 22-27, 2022 c⃝2022 Association for Computational Linguistics DuReadervis: A Chinese Dataset for Open-domain Document Visual Question Answering Le Qi1∗, Shangwen Lv2, Hongyu Li2, Jing Liu2, Yu Zhang1†, Qiaoqiao She2, Hua Wu2, Haifeng Wang2 and Ting Liu1 1Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology, Harbin, China 2Baidu Inc. {lqi, zhangyu, tliu}@ir.hit.edu.cn {lvshangwen, lihongyu04, liujing46, sheqiaoqiao, wu_hua, wanghaifeng}@baidu.com Abstract Open-domain question answering has been used in a wide range of applications, such as web search and enterprise search, which usually takes clean texts extracted from various formats of documents (e.g., web pages, PDFs, or Word documents) as the information source. However, designing different text extraction approaches is time-consuming and not scalable. In order to reduce human cost and improve the scalability of QA systems, we propose and study an Open-domain Document Visual Question Answering (Open-domain DocVQA) task, which requires answering questions based on a collection of document images directly instead of only document texts, utilizing layouts and visual features additionally. To advance this task, we introduce the first Chinese Open-domain DocVQA dataset called DuReadervis, containing about 15K question-answering pairs and 158K document images from the Baidu search engine. There are three main challenges in DuReadervis: (1) long document understanding, (2) noisy texts, and (3) multi-span answer extraction. The extensive experiments that the dataset is challenging.demonstrate Addition- ally, we propose a simple approach that incorporates the layout and visual features, and the experimental results show the ef- fectiveness of the proposed approach. The dataset and code will be publicly available at https://github.com/baidu/DuReader/tree/master/ DuReader-vis. 1 Introduction Open-domain Question Answering (Open-domain QA) is a task that requires answering questions based on a collection of document texts. It has been used in a wide range of applications, such as web search (He et al., 2018; Nguyen et al., 2016; ∗ The work was done when Le Qi was doing internship at Baidu. † Corresponding author. Web-pages in Source 1 Web-pages in Source 2 Answer Retrieve & QA Question Texts Extractor 1 PDFs in Source 1 Extractor 2 Documents in other Formats Extractor 3 Extractor n … … (a) The procedure in open-domain QA, which first utilizes different content extraction methods to get textual contents. Images Universal Extractor Texts Layouts Answer Retrieve & QA Question Web-pages in Source 1 Web-pages in Source 2 PDFs in Source 1 Document in other Formats … (b) The procedure in open-domain DocVQA, which utilizes a universal extractor to get textual contents and layout informa- tion. Figure 1: The procedure comparison between open- domain QA and open-domain DocVQA. Chen et al., 2017a), enterprise QA (Castelli et al., 2020), biomedical QA (Levy et al., 2021), etc. The typical procedure of an open-domain QA system can be summarized in Figure 1(a). It needs first to design specific text extraction methods for real-world documents in different formats (e.g., PDFs, web pages, scanned documents, etc.), and ex- tract certain text contents from them (e.g. the main body …

Content truncated to 500 words