Towards Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning

Proceedings of the 28th International Conference on Computational Linguistics, pages 2062–2072 Barcelona, Spain (Online), December 8-13, 2020 2062 Towards Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning Weipeng Huang∗ Xingyi Cheng∗† Kunlong Chen Taifeng Wang Wei Chu Ant Group {weipeng.hwp,fanyin.cxy, kunlong.ckl,taifeng.wang,weichu.cw}@antgroup.com Abstract The ambiguous annotation criteria lead to divergence of Chinese Word Segmentation (CWS) datasets in various granularities. Multi-criteria Chinese word segmentation aims to capture vari- ous annotation criteria among datasets and leverage their common underlying knowledge. In this paper, we propose a domain adaptive segmenter to exploit diverse criteria of various datasets. Our model is based on Bidirectional Encoder Representations from Transformers (BERT), which is responsible for introducing open-domain knowledge. Private and shared projection layers are proposed to capture domain-specific knowledge and common knowledge, respectively. We also optimize computational efficiency via distillation, quantization, and compiler optimization. Ex- periments show that our segmenter outperforms the previous state of the art (SOTA) models on 10 CWS datasets with superior efficiency. 1 Introduction Chinese Word Segmentation (CWS) is typically regarded as a low-level NLP task. Unlike English and French that uses the space token to separate the words, Chinese is a kind of polysynthetic languages where compounds are developed from indigenous morphemes (Jernudd and Shapiro, 2011; Gong et al., 2017). The ambiguous distinction between morphemes and compound words leads to the cognitive divergence of word concepts. Consequently, the labeled datasets seriously diverge due to the annotation inconsistency, resulting in multi-grained compounds. As shown in Table 1, given a sentence “刘国梁赢 得世界冠军” (Liu Guoliang wins the world championship), the two commonly used corpora, i.e., PKU’s People’s Daily (PKU) and Penn Chinese Treebank (CTB), use different segmentation criteria. In practice, a segmenter usually provides multiple configures with different granularities to better serve various downstream tasks. Fine-grained criterion is able to reduce the vocabulary, thereby relieves the sparseness issue. On the other hand, coarse-grained words provide more specific meanings, which may benefit the domain-specific tasks. In recent years, several multi-criteria learning methods for CWS have been proposed to explore the common knowledge of heterogeneous datasets. By utilizing the information across all corpora, multi- criteria learning methods can boost the out-of-vocabulary (OOV) recalls as well as practical perfor- mance (Qiu et al., 2013; Chao et al., 2015; Chen et al., 2017). Despite its effectiveness, there still are three unresolved issues. (1) Even with multiple datasets, the data is still limited to provide adequate linguistic knowledge. (2) Learning from a dataset is likely to hurt the others as the segmentation follows inconsistent criteria. (3) The advanced models, e.g., Bi-LSTM-CRF (Ma et al., 2018), are computation- ally expensive. They are based on the recurrent neural networks (RNNs). Since RNNs are auto-regressive and the computation cannot be completed in parallel, their applications are usually limited due to the poor computational efficiency. In fact, the inference speed is heavily required for the CWS system as it serves ∗Equal contribution. †Corresponding author. This work is licensed under a Creative Commons Attribution 4.0 International Licence. Licence details: http:// creativecommons.org/licenses/by/4.0/. 2063 …

Content truncated to 500 words