Convolutional Neural Network with Word Embeddings for Chinese Word Segmentation

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 163–172, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Convolutional Neural Network with Word Embeddings for Chinese Word Segmentation Chunqi Wang1,2 and Bo Xu1 1Institute of Automation, Chinese Academy of Sciences 2University of Chinese Academy of Sciences chqiwang@126.com, xubo@ia.ac.cn Abstract Character-based sequence labeling frame- work is flexible and efficient for Chi- nese word segmentation (CWS). Recently, many character-based neural models have been applied to CWS. While they obtain good performance, they have two obvious weaknesses. The first is that they heav- ily rely on manually designed bigram fea- ture, i.e. they are not good at captur- ing n-gram features automatically. The second is that they make no use of full word information. For the first weakness, we propose a convolutional neural model, which is able to capture rich n-gram fea- tures without any feature engineering. For the second one, we propose an effective approach to integrate the proposed model with word embeddings. We evaluate the model on two benchmark datasets: PKU and MSR. Without any feature engineer- ing, the model obtains competitive per- formance — 95.7% on PKU and 97.3% on MSR. Armed with word embeddings, the model achieves state-of-the-art perfor- mance on both datasets — 96.5% on PKU and 98.0% on MSR, without using any ex- ternal labeled resource. 1 Introduction Unlike English and other western languages, most east Asian languages, including Chinese, are writ- ten without explicit word delimiters. However, most natural language processing (NLP) applica- tions are word-based. Therefore, word segmen- tation is an essential step for processing those languages. CWS is often treated as a character- based sequence labeling task (Xue et al., 2003; Peng et al., 2004). Figure 1 gives an intuitive �� �� �� �� �� �� �� �� �� �� � �� �� �� �� �� ���� ������� ���� ����������� ���� � Figure 1: Chinese word segmentation as a sequence label- ing task. This figure presents the common BMES (Begining, Middle, End, Singleton) tagging scheme. explaination. Linear models, such as Maximum Entropy (ME) (Berger et al., 1996) and Con- ditional Random Fields (CRF) (Lafferty et al., 2001), have been widely used for sequence label- ing tasks. However, they often depend heavily on well-designed hand-crafted features. Recently, neural networks have been widely used for NLP tasks. Collobert et al. (2011) pro- posed a unified neural architecture for various se- quence labeling tasks. Instead of exploiting hand- crafted input features carefully optimized for each task, their system learns internal representations automatically. As for CWS, there are a series of works, which share the main idea with Col- lobert et al. (2011) but vary in the network ar- chitecture. In particular, feed-forward neural net- work (Zheng et al., 2013), tensor neural network (Pei et al., 2014), recursive neural network (Chen et al., 2015a), long-short term memory (LSTM) (Chen et al., 2015b), as well as the combination of LSTM and recursive neural network (Xu and Sun, 2016) have been used to derive …

Content truncated to 500 words