Adaptive Multi-Task Transfer Learning for Chinese Word Segmentation in Medical Text

Proceedings of the 27th International Conference on Computational Linguistics, pages 3619–3630 Santa Fe, New Mexico, USA, August 20-26, 2018. 3619 Adaptive Multi-Task Transfer Learning for Chinese Word Segmentation in Medical Text Junjie Xing Kenny Q. Zhu Shaodian Zhang Shanghai Jiao Tong University {jjxing@,kzhu@cs.,shaodian@apex.}sjtu.edu.cn Abstract Chinese word segmentation (CWS) trained from open source corpus faces dramatic performance drop when dealing with domain text, especially for a domain with lots of special terms and diverse writing styles, such as the biomedical domain. However, building domain-specific CWS requires extremely high annotation cost. In this paper, we propose an approach by exploiting domain-invariant knowledge from high resource to low resource domains. Extensive experiments show that our model achieves consistently higher accuracy than the single-task CWS and other transfer learning baselines, especially when there is a large disparity between source and target domains. 1 Introduction Chinese word segmentation (CWS) is a fundamental task for Chinese natural language processing (NLP). Most state-of-art methods are based on statistical supervised learning and neural networks. They all rely heavily on human-annotated data, which is a time-consuming and expensive work. Specially, for domain CWS, e.g., medical field , the annotation expense is even higher because only domain experts are qualified for the work. Moreover, CWS tools trained from open source datasets, e.g., SIGHAN20051, face a significance performance drop when dealing with domain text. The ambiguity caused by domain terms and writing style makes it extremely difficult to train a universal CWS tool. As shown in Table 1, given a medical term “高铁血红蛋白血症” (methemoglobinemia), Chinese medical experts would annotate it as “高/铁/血红 蛋白/血症”, which means anemia caused by hemoglobin with “high iron” (in Chinese, means iron with valence of 3), corresponding to the morphology of “Methemoglobinemia”. “PKU” stands for a model trained on PKU’s People’s Daily corpus, we can see that after segmentation, the word “铁血” (jagged) is treated as one word, which is wrong semantically. Also, another popular Chinese CWS tool Jieba 2 mistakenly puts the characters “高” and “铁” together, which stands for the high-speed bullet train in China. CWS tool 高 高 高铁 铁 铁血 血 血红 红 红蛋 蛋 蛋白 白 白血 血 血症 症 症 PKU 高 铁血 红蛋白 血症 high jagged albumen anemia Jieba 高铁 血红蛋白 血症 train hemoglobin anemia Medical 高 铁 血红蛋白 血症 high iron hemoglobin anemia Table 1: Medical CWS ambiguity with CWS tools. PKU stands for a model trained on PKU dataset. In summary, domain specific CWS task poses significant challenges because: Kenny Q. Zhu is the corresponding author. This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http://creativecommons.org/licenses/by/4.0/ 1http://sighan.cs.uchicago.edu/bakeoff2005/ 2https://github.com/fxsjy/jieba 3620 1. Tools built on open source annotated corpus works badly on domain specific CWS. 2. Annotated domain data is scarce due to high cost. 3. How to leverage open source annotated data despite their generality is an open question. Recently, efforts have been made to exploit open source (high resource) data to improve the perfor- mance of domain specific (low resource) tasks and decrease …

Content truncated to 500 words