A Concise Model for Multi-Criteria Chinese Word Segmentation with Transformer Encoder

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2887–2897 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 2887 A Concise Model for Multi-Criteria Chinese Word Segmentation with Transformer Encoder Xipeng Qiu∗, Hengzhi Pei, Hang Yan, Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University {xpqiu, hzpei16, hyan19, xjhuang}@fudan.edu.cn Abstract Multi-criteria Chinese word segmentation (MCCWS) aims to exploit the relations among the multiple heterogeneous segmentation cri- teria and further improve the performance of each single criterion. Previous work usually regards MCCWS as different tasks, which are learned together under the multi-task learn- ing framework. In this paper, we propose a concise but effective unified model for MC- CWS, which is fully-shared for all the cri- teria. By leveraging the powerful ability of the Transformer encoder, the proposed unified model can segment Chinese text according to a unique criterion-token indicating the output criterion. Besides, the proposed unified model can segment both simplified and traditional Chinese and has an excellent transfer capabil- ity. Experiments on eight datasets with differ- ent criteria show that our model outperforms our single-criterion baseline model and other multi-criteria models. Source codes of this pa- per are available on Github1. 1 Introduction Chinese word segmentation (CWS) is a prelimi- nary step to process Chinese text. The mainstream CWS methods regard CWS as a character-based se- quence labeling problem, in which each character is assigned a label to indicate its boundary infor- mation. Recently, various neural models have been explored to reduce efforts of the feature engineer- ing (Chen et al., 2015a,b; Qun et al., 2020; Wang and Xu, 2017; Kurita et al., 2017; Ma et al., 2018). Recently, Chen et al. (2017) proposed multi- criteria Chinese word segmentation (MCCWS) to effectively utilize the heterogeneous resources with different segmentation criteria. Specifically, they regard each segmentation criterion as a single ∗Corresponding author. 1https://github.com/acphile/MCCWS Corpora Lin Dan won the championship CTB 林丹 赢得 总冠军 PKU 林 丹 赢得 总 冠军 MSRA 林丹 赢得 总 冠军 Table 1: Illustration of different segmentation criteria. task under the framework of multi-task learning, where a shared layer is used to extract the criteria- invariant features, and a private layer is used to extract the criteria-specific features. However, it is unnecessary to use a specific pri- vate layer for each criterion. These different criteria often have partial overlaps. For the example in Ta- ble 1, the segmentation of “林丹(Lin Dan)” is the same in CTB and MSRA criteria, and the segmen- tation of “总|冠军(the championship)” is the same in PKU and MSRA criteria. All these three crite- ria have the same segmentation for the word “赢 得(won)”. Although these criteria are inconsistent, they share some partial segmentation. Therefore, it is interesting to use a unified model for all the criteria. At the inference phase, a criterion-token is taken as input to indicate the predict segmentation criterion. Following this idea, Gong et al. (2018) used multiple LSTMs and a criterion switcher at every position to automatically …

Content truncated to 500 words