Enhancing Chinese Word Segmentation via Pseudo Labels for Practicability

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4369–4381 August 1–6, 2021. ©2021 Association for Computational Linguistics 4369 Enhancing Chinese Word Segmentation via Pseudo Labels for Practicability Kaiyu Huang1 Junpeng Liu1 Degen Huang1∗ Deyi Xiong2,3 Zhuang Liu4 Jinsong Su5 1Dalian University of Technology 2Tianjin University 3Global Tone Communication Technology Co., Ltd. 4Dongbei University of Finance and Economics 5Xiamen University {kaiyuhuang,liujunpeng nlp}@mail.dlut.edu.cn huangdg@dlut.edu.cn liuzhuang@dufe.edu.cndyxiong@tju.edu.cn jssu@xmu.edu.cn Abstract Pre-trained language models (e.g., BERT) sig- nificantly alleviate two traditional challeng- ing problems for Chinese word segmentation (CWS): segmentation ambiguity and out-of- vocabulary (OOV) words. However, such im- provements are usually achieved on traditional benchmark datasets and not close to an im- portant goal of CWS: practicability (i.e., low complexity as a standalone task and high ben- eficiality to downstream tasks). To make a trade-off between traditional evaluation and practicability for CWS, we propose a semi- supervised neural method via pseudo labels. The neural method consists of a teacher model and a student model, which distills knowl- edge from unlabeled data to the student model so as to improve both in-domain and out-of- domain CWS. Experiments show that our pro- posed method can not only keep the practi- cability of the lightweight student model but also improve the performance of segmentation effectively. We also evaluate a range of het- erogeneous neural architectures of CWS on downstream Chinese NLP tasks. Results of further experiments demonstrate that our pro- posed segmenter is reliable and practical as a pre-processing step of the downstream NLP tasks at the minimum cost.1 1 Introduction Natural language processing (NLP) tasks often leverage word-level features to exploit lexical knowledge. Segmenting a sentence into a sequence of words, especially for languages without explicit word boundaries (e.g., Chinese) not only extracts lexical features, but also shortens the length of the sentence to be processed. Thus, word segmenta- tion, detecting word boundaries, is a crucial pre- ∗Corresponding author 1Our code is available at https://github.com/ koukaiu/dlut-nihao processing task for many NLP tasks. In this aspect, Chinese word segmentation (CWS) is widely ac- knowledged as an essential task for Chinese NLP. CWS has made substantial progress in recent studies on several benchmarks, which is reported by Huang and Zhao (2007) and Zhao et al. (2019). In particular, pretrained language models (PLMs), like BERT (Devlin et al., 2019), have established new state-of-the-art in sequence labeling (Meng et al., 2019). Various fine-tuning methods have been proposed to improve the performance of in- domain and cross-domain CWS based on PLMs (Huang et al., 2020; Tian et al., 2020). The two challenging problems in CWS, segmentation am- biguity and out-of-vocabulary (OOV) words, have been significantly mitigated by PLM-based meth- ods that are fine-tuned on large-scale annotated CWS corpora. Such methods are even reaching human performance on benchmarks. Nevertheless, CWS is more valuable as a prelude for downstream NLP tasks than as a standalone task. Intrinsic eval- uation of CWS on benchmark datasets only exam- ines the effectiveness of current neural methods on word boundary detection. To better apply CWS in …

Content truncated to 500 words