Knowledge-Empowered Representation Learning for Chinese Medical Reading Comprehension: Task, Model and Resources

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2237–2249 August 1–6, 2021. ©2021 Association for Computational Linguistics 2237 Knowledge-Empowered Representation Learning for Chinese Medical Reading Comprehension: Task, Model and Resources Taolin Zhang1,2∗, Chengyu Wang2∗, Minghui Qiu2, Bite Yang3 Zerui Cai4, Xiaofeng He4†, Jun Huang2 1 School of Software Engineering, East China Normal University 2 Alibaba Group 3 DXY 4 School of Computer Science and Technology, East China Normal University zhangtl0519@gmail.com, zrcai flow@126.com, yangbt@dxy.cn {chengyu.wcy, minghui.qmh, huangjun.hj}@alibaba-inc.com hexf@cs.ecnu.edu.cn Abstract Machine Reading Comprehension (MRC) aims to extract answers to questions given a passage, which has been widely studied recently especially in open domains. How- ever, few efforts have been made on closed- domain MRC, mainly due to the lack of large- scale training data. In this paper, we intro- duce a multi-target MRC task for the med- ical domain, whose goal is to predict an- swers to medical questions and the correspond- ing support sentences from medical informa- tion sources simultaneously, in order to en- sure the high reliability of medical knowl- edge serving. A high-quality dataset (more than 18k samples) is manually constructed for the purpose, named Multi-task Chinese Medical MRC dataset (CMedMRC), with de- tailed analysis conducted. We further pro- pose a Chinese medical BERT model for the task (CMedBERT), which fuses medical knowledge into pre-trained language models by the dynamic fusion mechanism of hetero- geneous features and the multi-task learning strategy. Experiments show that CMedBERT consistently outperforms strong baselines by fusing context-aware and knowledge-aware to- ken representations.1 1 Introduction Machine Reading Comprehension (MRC) has be- come a popular task in NLP, aiming to understand a given passage and answer the relevant questions. With the wide availability of MRC datasets (Ra- jpurkar et al., 2016; He et al., 2018; Cui et al., 2019) and deep learning models (Yu et al., 2018; Ding et al., 2019) (including pre-trained language mod- els such as BERT (Devlin et al., 2019)), significant progress has been made. ∗T. Zhang and C. Wang contributed equally to this work. †Corresponding author. 1The code and dataset will be available at https:// github.com/MatNLP/CMedMRC Despite the success, a majority of MRC research has focused on open domains. For specific do- mains, however, the construction of high-quality MRC datasets, together with the design of corre- sponding models is considerably deficient (Welbl et al., 2017, 2018). The causes behind this phe- nomenon are threefold. Take the medical domain as an example. i) Data annotators are required to have medical backgrounds with high standards. Hence, simple crowd-sourcing (Rajpurkar et al., 2016; Cui et al., 2019) often leads to poor annotation results. ii) Due to the domain sensitivity, people are more concerned about the reliability of the information sources where the answers are extracted, and the explainability of the answers themselves (Lee et al., 2014; Dalmer, 2017). This is fundamentally dif- ferent from the task requirements of open-domain MRC. iii) From the perspective of model learning, it is difficult for pre-trained language models to understand the meaning of the questions and pas- …

Content truncated to 500 words