Accurate Linear-Time Chinese Word Segmentation via Embedding Matching

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1733–1743, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Accurate Linear-Time Chinese Word Segmentation via Embedding Matching Jianqiang Ma SFB 833 and Department of Linguistics University of Tübingen, Germany jma@sfs.uni-tuebingen.de Erhard Hinrichs SFB 833 and Department of Linguistics University of Tübingen, Germany eh@sfs.uni-tuebingen.de Abstract This paper proposes an embedding match- ing approach to Chinese word segmenta- tion, which generalizes the traditional se- quence labeling framework and takes ad- vantage of distributed representations. The training and prediction algorithms have linear-time complexity. Based on the pro- posed model, a greedy segmenter is de- veloped and evaluated on benchmark cor- pora. Experiments show that our greedy segmenter achieves improved results over previous neural network-based word seg- menters, and its performance is competi- tive with state-of-the-art methods, despite its simple feature set and the absence of ex- ternal resources for training. 1 Introduction Chinese sentences are written as character se- quences without word delimiters, which makes word segmentation a prerequisite of Chinese lan- guage processing. Since Xue (2003), most work has formulated Chinese word segmentation (CWS) as sequence labeling (Peng et al., 2004) with char- acter position tags, which has lent itself to struc- tured discriminative learning with the benefit of allowing rich features of segmentation configura- tions, including (i) context of character/word n- grams within local windows, (ii) segmentation his- tory of previous characters, or the combinations of both. These feature-based models still form the backbone of most state-of-the art systems. Nevertheless, many feature weights in such models are inevitably poorly estimated because the number of parameters is so large with respect to the limited amount of training data. This has mo- tivated the introduction of low-dimensional, real- valued vectors, known as embeddings, as a tool to deal with the sparseness of the input. Em- beddings allow linguistic units appearing in sim- ilar contexts to share similar vectors. The suc- cess of embeddings has been observed in many NLP tasks. For CWS, Zheng et al. (2013) adapted Collobert et al. (2011) and uses character embed- dings in local windows as input for a two-layer net- work. The network predicts individual character position tags, the transitions of which are learned separately. Mansur et al. (2013) also developed a similar architecture, which labels individual char- acters and uses character bigram embeddings as additional features to compensate the absence of sentence-level modeling. Pei et al. (2014) im- proved upon Zheng et al. (2013) by capturing the combinations of context and history via a tensor neural network. Despite their differences, these CWS ap- proaches are all sequence labeling models. In such models, the target character can only influence the prediction as features. Consider the the segmen- tation configuration in (1), where the dot appears before the target character in consideration and the box (2) represents any character that can occur in the configuration. In that example, the known his- tory is that …

Content truncated to 500 words