Exploring Representations from Unlabeled Data with Co-training for Chinese Word Segmentation

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 311–321, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Exploring Representations from Unlabeled Data with Co-training for Chinese Word Segmentation Longkai Zhang Houfeng Wang∗ Xu Sun Mairgup Mansur Key Laboratory of Computational Linguistics (Peking University) Ministry of Education, China zhlongk@qq.com, wanghf@pku.edu.cn, xusun@pku.edu.cn, mairgup@gmail.com, Abstract Nowadays supervised sequence labeling models can reach competitive performance on the task of Chinese word segmenta- tion. However, the ability of these mod- els is restricted by the availability of an- notated data and the design of features. We propose a scalable semi-supervised fea- ture engineering approach. In contrast to previous works using pre-defined task- specific features with fixed values, we dy- namically extract representations of label distributions from both an in-domain cor- pus and an out-of-domain corpus. We update the representation values with a semi-supervised approach. Experiments on the benchmark datasets show that our approach achieve good results and reach an f-score of 0.961. The feature engineer- ing approach proposed here is a general iterative semi-supervised method and not limited to the word segmentation task. 1 Introduction Chinese is a language without natural word delimiters. Therefore, Chinese Word Segmen- tation (CWS) is an essential task required by further language processing. Previous research shows that sequence labeling models trained on labeled data can reach competitive accuracy on the CWS task, and supervised models are more accurate than unsupervised models (Xue, 2003; Low et al., 2005). However, the resource of man- ually labeled training corpora is limited. There- fore, semi-supervised learning has become one ∗Corresponding author of the most natural forms of training for CWS. Traditional semi-supervised methods focus on adding new unlabeled instances to the training set by a given criterion. The possible mislabeled instances, which are introduced from the auto- matically labeled raw data, can hurt the per- formance and not easy to exclude by setting a sound selecting criterion. In this paper, we propose a simple and scal- able semi-supervised strategy that works by pro- viding semi-supervision at the level of represen- tation. Previous works mainly assume that con- text features are helpful to decide the potential label of a character. However, when some of the context features do not appear in the training corpus, this assumption may fail. An example is shown in table 1. Although the context of “水” and “篮” is totally different, they share a homo- geneous structure as “verb-noun”. Therefore. A much better way is to map the context informa- tion to a kind of representation. More precisely, the mapping should let the similar contexts map to similar representations, while let the distinct contexts map to distinct representations. 吃水果 打篮球 Label B B Character 吃 水 果 打 篮 球 Context C-1= 吃 C-1= 打 Features C0= 水 C0= 篮 C1= 果 C1= 球 Table 1: Example of the context of “水” in “吃水 果 (Eat fruits)” and the context of “篮” in “打篮球 (Play basketball)” We use the label distribution information that …

Content truncated to 500 words