A Span-Extraction Dataset for Chinese Machine Reading Comprehension

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5883–5889, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5883 A Span-Extraction Dataset for Chinese Machine Reading Comprehension Yiming Cui†‡, Ting Liu†, Wanxiang Che†, Li Xiao‡, Zhipeng Chen‡, Wentao Ma‡§, Shijin Wang‡§, Guoping Hu‡ †Research Center for Social Computing and Information Retrieval (SCIR), Harbin Institute of Technology, Harbin, China ‡State Key Laboratory of Cognitive Intelligence, iFLYTEK Research, China §iFLYTEK AI Research (Hebei), Langfang, China †{ymcui,tliu,car}@ir.hit.edu.cn ‡§{ymcui,lixiao3,zpchen,wtma,sjwang3,gphu}@iflytek.com Abstract Machine Reading Comprehension (MRC) has become enormously popular recently and has attracted a lot of attention. However, the exist- ing reading comprehension datasets are mostly in English. In this paper, we introduce a Span- Extraction dataset for Chinese machine read- ing comprehension to add language diversities in this area. The dataset is composed by near 20,000 real questions annotated on Wikipedia paragraphs by human experts. We also anno- tated a challenge set which contains the ques- tions that need comprehensive understanding and multi-sentence inference throughout the context. We present several baseline systems as well as anonymous submissions for demon- strating the difficulties in this dataset. With the release of the dataset, we hosted the Sec- ond Evaluation Workshop on Chinese Ma- chine Reading Comprehension (CMRC 2018). We hope the release of the dataset could fur- ther accelerate the Chinese machine reading comprehension research.1 1 Introduction To read and comprehend natural languages is the key to achieve advanced artificial intelligence. Machine Reading Comprehension (MRC) aims to comprehend the context of given articles and answer the questions based on them. Various types of machine reading comprehension datasets have been proposed, such as cloze-style reading comprehension (Hermann et al., 2015; Hill et al., 2015; Cui et al., 2016), span-extraction reading comprehension (Rajpurkar et al., 2016; Trischler et al., 2016), open-domain reading comprehension (Nguyen et al., 2016; He et al., 2017), reading comprehension with multiple-choice (Richardson et al., 2013; Lai et al., 2017), etc. Along with 1Resources are available: https://github.com/ ymcui/cmrc2018. the development of the reading comprehension dataset, various neural network approaches have been proposed and made a big advancement in this area (Kadlec et al., 2016; Cui et al., 2017; Dhingra et al., 2017; Wang and Jiang, 2016; Xiong et al., 2016; Liu et al., 2017; Wang et al., 2017; Hu et al., 2018; Wang et al., 2018; Yu et al., 2018). We also have seen various efforts on the con- struction of Chinese machine reading compre- hension datasets. In cloze-style reading compre- hension, Cui et al. (2016) proposed a Chinese cloze-style reading comprehension dataset: Peo- ple’s Daily & Children’s Fairy Tale. To add dif- ficulties to the dataset, along with the automati- cally generated evaluation sets (development and test), they also release a human-annotated evalua- tion set. Later, Cui et al. (2018) propose another dataset, which is gathered from children’s reading material. To add more diversity and for further in- vestigation on transfer learning, they also provide another evaluation …

Content truncated to 500 words