WebSRC: A Dataset for Web-Based Structural Reading Comprehension

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4173–4185 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4173 WebSRC: A Dataset for Web-Based Structural Reading Comprehension Xingyu Chen, Zihan Zhao, Lu Chen∗, Jiabao JI, Danyang Zhang, Ao Luo, Yuxuan Xiong and Kai Yu∗ X-LANCE Lab, Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University Shanghai Jiao Tong University, Shanghai, China State Key Lab of Media Convergence Production Technology and Systems, Beijing, China galaxychen@sjtu.edu.cn, zhao_mengxin@sjtu.edu.cn, chenlusz@sjtu.edu.cn, {sjcs_jijiabao, zhang-dy20, wenzelmetternich, xiongyx, kai.yu}@sjtu.edu.cn Abstract Web search is an essential way for humans to obtain information, but it’s still a great challenge for machines to understand the con- tents of web pages. In this paper, we intro- duce the task of structural reading compre- hension (SRC) on web. Given a web page and a question about it, the task is to find the answer from the web page. This task re- quires a system not only to understand the semantics of texts but also the structure of the web page. Moreover, we proposed Web- SRC, a novel Web-based Structural Reading Comprehension dataset. WebSRC consists of 400K question-answer pairs, which are col- lected from 6.4K web pages. Along with the QA pairs, corresponding HTML source code, screenshots, and metadata are also provided in our dataset. Each question in WebSRC re- quires a certain structural understanding of a web page to answer, and the answer is either a text span on the web page or yes/no. We eval- uate various baselines on our dataset to show the difficulty of our task. We also investigate the usefulness of structural information and vi- sual features. Our dataset and baselines have been publicly available1. 1 Introduction Web pages are the most common source of human knowledge and daily information. With the help of modern search engines, people can easily locate web pages and find information by simply typing some keywords. However, traditional search en- gines only retrieve web pages related to the query and highlight the possible answers (Chen, 2018), they can’t understand the web pages and answer the query based on contents. The rapid development of question answering systems and knowledge graphs enables search engines to answer simple questions directly (Chakraborty et al., 2019), but they still ∗The corresponding authors are Lu Chen and Kai Yu. 1https://x-lance.github.io/WebSRC/ Figure 1: Examples for a web page. (a) is the original web page. (b) is the HTML code for the content in the red box. Each HTML tag begins with a starting tag and ends with a closing tag (with a slash in tag). <td> stands for a table cell and <span> stands for a content span. (c) shows the text extracted from the web page. (d) contains some sample questions for the web page. fail to perform question answering on arbitrary web pages. The difficulty lies in the variety of web pages and the complexity of the web layouts, which requires a system not only to …

Content truncated to 500 words