A Joint Multiple Criteria Model in Transfer Learning for Cross-domain Chinese Word Segmentation

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3873–3882, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3873 A Joint Multiple Criteria Model in Transfer Learning for Cross-domain Chinese Word Segmentation Kaiyu Huang, Degen Huang∗, Zhuang Liu and Fengran Mo School of Computer Science, Dalian University of Technology {kaiyuhuang,zhuangliu,fengranmo}@mail.dlut.edu.cn huangdg@dlut.edu.cn Abstract Word-level information is important in natu- ral language processing (NLP), especially for the Chinese language due to its high linguis- tic complexity. Chinese word segmentation (CWS) is an essential task for Chinese down- stream NLP tasks. Existing methods have al- ready achieved a competitive performance for CWS on large-scale annotated corpora. How- ever, the accuracy of the method will drop dra- matically when it handles an unsegmented text with lots of out-of-vocabulary (OOV) words. In addition, there are many different segmen- tation criteria for addressing different require- ments of downstream NLP tasks. Excessive amounts of models with saving different crite- ria will generate the explosive growth of the to- tal parameters. To this end, we propose a joint multiple criteria model that shares all parame- ters to integrate different segmentation criteria into one model. Besides, we utilize a transfer learning method to improve the performance of OOV words. Our proposed method is evalu- ated by designing comprehensive experiments on multiple benchmark datasets (e.g., Bake- off 2005, Bakeoff 2008 and SIGHAN 2010). Our method achieves the state-of-the-art per- formances on all datasets. Importantly, our method also shows a competitive practicability and generalization ability for the CWS task. 1 Introduction In the extensive researches on natural language processing (NLP), most of the tasks are based on word-level methods because word is the smallest linguistic unit in natural languages. It has rich feature information. However, the situation is to- tally different when dealing with the Chinese lan- guage. There is not clearly delimiter between Chi- nese words, instead the blank space is regarded as a delimiter in most western languages. Different ∗Corresponding author Corpora Zhang Xiao Fan attend a tournament PKU 张 小凡 参加 比武 大会 MSRA 张小凡 参加 比武大会 Zhuxian 张小凡 参加 比武 大会 Table 1: Illustration of different segmentation criteria on three popular datasets segmentation results may lead to different feature information. Thus, Chinese word segmentation (CWS) is an essential task, which will significantly affect the effectiveness of downstream Chinese NLP tasks. Recently, the approaches for CWS have already achieved a good performance in large- scale annotated corpora, as reported by related re- searches (Huang and Zhao, 2007; Zhao et al., 2019). Most of the effective approaches fall into two ma- jor research fields: the statistical machine learning method and the neural network method. The for- mer is mainly based on Conditional Random Fields (CRF), which is considered as the most effective statistical machine learning method for CWS (Zhao and Kit, 2008; Zhao et al., 2010). However, the statistical machine learning method always heav- ily relies on hand-craft features. To minimize the efforts in feature engineering, more and more re- searches are focus on neural …

Content truncated to 500 words