Attention Is All You Need for Chinese Word Segmentation

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3862–3872, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3862 Attention Is All You Need for Chinese Word Segmentation Sufeng Duan1,2,3, Hai Zhao1,2,3∗ 1Department of Computer Science and Engineering, Shanghai Jiao Tong University 2Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, China 3MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University 1140339019dsf@sjtu.edu.cn, zhaohai@cs.sjtu.edu.cn Abstract Taking greedy decoding algorithm as it should be, this work focuses on further strengthen- ing the model itself for Chinese word seg- mentation (CWS), which results in an even more fast and more accurate CWS model. Our model consists of an attention only stacked encoder and a light enough decoder for the greedy segmentation plus two highway con- nections for smoother training, in which the encoder is composed of a newly proposed Transformer variant, Gaussian-masked Direc- tional (GD) Transformer, and a biaffine atten- tion scorer. With the effective encoder de- sign, our model only needs to take unigram features for scoring. Our model is evaluated on SIGHAN Bakeoff benchmark datasets. The experimental results show that with the high- est segmentation speed, the proposed model achieves new state-of-the-art or comparable performance against strong baselines in terms of strict closed test setting. 1 Introduction Chinese word segmentation (CWS) is the task of delimiting word boundaries in a sentence, as a ba- sic and essential task for Chinese and many other East Asian languages which are written without explicit word delimiters, and thus different from alphabetical languages like English. Learning from an annotated corpus with segmen- tation, the CWS task may be generally modeled as a decoder which performs segmentation based on a scoring module in terms of contextual feature based representations. Table 1 summarizes typi- cal CWS models according to their decoding ways. ∗Corresponding author. This paper was partially sup- ported by National Key Research and Development Program of China (No. 2017YFB0304100), Key Projects of Na- tional Natural Science Foundation of China (U1836222 and 61733011), Huawei-SJTU long term AI project, Cutting-edge Machine reading comprehension and language model. Markov models such as (Ng and Low, 2004) and (Zheng et al., 2013) depend on the maximum en- tropy model or maximum entropy Markov model both with Viterbi decoding. Besides, conditional random field (CRF) or Semi-CRF for sequence la- beling has been used for both traditional and neural models though with different representations (Peng et al., 2004; Andrew, 2006; Wang and Xu, 2017; Ma et al., 2018). Recent neural CWS research have been con- cerned about the following three perspectives (Emerson, 2005). Decoder. As CWS is a kind of structure learn- ing task, the decoder module generally determines which type of detailed algorithm should be adopted for segmentation, also it may limit the capability of defining feature. As shown in Table 2, not all mod- els can support the word-level features as CWS is a task to predict word boundary. Thus recent works focus on finding …

Content truncated to 500 words