A Fine-Grained Domain Adaption Model for Joint Word Segmentation and POS Tagging

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3587–3598 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3587 A Fine-Grained Domain Adaption Model for Joint Word Segmentation and POS Tagging Peijie Jiang1 Dingkun Long Yueheng Sun2 Meishan Zhang1∗ Guangwei Xu Pengjun Xie 1School of New Media and Communication, Tianjin University, China 2College of Intelligence and Computing, Tianjin University, China {jzx555,yhs,zhangmeishan}@tju.edu.cn {longdingkun1993,ahxgwOnePiece,xpjandy}@gmail.com Abstract Domain adaption for word segmentation and POS tagging is a challenging problem for Chi- nese lexical processing. Self-training is one promising solution for it, which struggles to construct a set of high-quality pseudo train- ing instances for the target domain. Previ- ous work usually assumes a universal source- to-target adaption to collect such pseudo cor- pus, ignoring the different gaps from the tar- get sentences to the source domain. In this work, we start from joint word segmentation and POS tagging, presenting a fine-grained do- main adaption method to model the gaps ac- curately. We measure the gaps by one simple and intuitive metric, and adopt it to develop a pseudo target domain corpus based on fine- grained subdomains incrementally. A novel domain-mixed representation learning model is proposed accordingly to encode the multi- ple subdomains effectively. The whole pro- cess is performed progressively for both cor- pus construction and model training. Exper- imental results on a benchmark dataset show that our method can gain significant improve- ments over a vary of baselines. Extensive anal- yses are performed to show the advantages of our final domain adaption model as well. 1 Introduction Chinese Word Segmentation (CWS) and Part-Of- Speech (POS) tagging are two fundamental tasks for natural language processing (NLP) in Chinese (Emerson, 2005; Jin and Chen, 2008), serving as backbones for a number of downstream NLP tasks. The joint models of the two tasks can lead to bet- ter performance because they are closely-related and the pipeline models suffer from the error prop- agation problem (Ng and Low, 2004; Zhang and Clark, 2008; Wang et al., 2011; Zeng et al., 2013; Zhang et al., 2018; Tian et al., 2020a), which can be alleviated in the joint architecture. ∗Corresponding author. Target Raw Text Source Domain Target Domain Plain Self-Training Fine-Grained Adaption ST1 STn ST2 · · · Figure 1: The idea of fine-grained domain adaption. Currently, joint CWS and POS tagging has gained great achievements with BERT inputs (Tian et al., 2020a,b). Our preliminary results show that the F1-score of joint POS tagging can be close to 95% when the training and test corpus both belong to a standard newswire domain. Unfortunately, it is not always the case in real applications. The performance might be degraded dramatically when the source and target domains are highly different. Taken the ZhuXian (a novel from Internet) as an example (Zhang et al., 2014), the same model can only obtain an F1-score of 89% for POS tagging according to our results. It is a typical domain adaption problem targeted to joint CWS and POS tagging. Self-training could be …

Content truncated to 500 words