Semi-Supervised Chinese Word Segmentation Using Partial-Label Learning With Conditional Random Fields

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 90–98, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Semi-Supervised Chinese Word Segmentation Using Partial-Label Learning With Conditional Random Fields Fan Yang Nuance Communications Inc. fan.yang@nuance.com Paul Vozila Nuance Communications Inc. paul.vozila@nuance.com Abstract There is rich knowledge encoded in on- line web data. For example, punctua- tion and entity tags in Wikipedia data define some word boundaries in a sen- tence. In this paper we adopt partial-label learning with conditional random fields to make use of this valuable knowledge for semi-supervised Chinese word segmenta- tion. The basic idea of partial-label learn- ing is to optimize a cost function that marginalizes the probability mass in the constrained space that encodes this knowl- edge. By integrating some domain adap- tation techniques, such as EasyAdapt, our result reaches an F-measure of 95.98% on the CTB-6 corpus, a significant improve- ment from both the supervised baseline and a previous proposed approach, namely constrained decode. 1 Introduction A general approach for supervised Chinese word segmentation is to formulate it as a character se- quence labeling problem, to label each charac- ter with its location in a word. For example, Xue (2003) proposes a four-label scheme based on some linguistic intuitions: ‘B’ for the beginning character of a word, ‘I’ for the internal characters, ‘E’ for the ending character, and ‘S’ for single- character word. Thus the word sequence “洽谈会 很 成功” can be turned into a character sequence with labels as 洽\B 谈\I 会\E 很\S 成\B 功\E. A machine learning algorithm for sequence label- ing, such as conditional random fields (CRF) (Laf- ferty et al., 2001), can be applied to the labelled training data to learn a model. Labelled data for supervised learning of Chi- nese word segmentation, however, is usually ex- pensive and tends to be of a limited amount. Re- searchers are thus interested in semi-supervised learning, which is to make use of unlabelled data to further improve the performance of supervised learning. There is a large amount of unlabelled data available, for example, the Gigaword corpus in the LDC catalog or the Chinese Wikipedia on the web. Faced with the large amount of unlabelled data, an intuitive idea is to use self-training or EM, by first training a baseline model (from the supervised data) and then iteratively decoding the unlabelled data and updating the baseline model. Jiao et al. (2006) and Mann and McCallum (2007) further propose to minimize the entropy of the predicted label distribution on unlabeled data and use it as a regularization term in CRF (i.e. entropy reg- ularization). Beyond these ideas, Liang (2005) and Sun and Xu (2011) experiment with deriv- ing a large set of statistical features such as mu- tual information and accessor variety from un- labelled data, and add them to supervised dis- criminative training. Zeng et al. (2013b) experi- ment with graph propagation to extract informa- tion from unlabelled data to regularize the CRF training. Yang and Vozila …

Content truncated to 500 words