Discriminative Learning with Natural Annotations: Word Segmentation as a Case Study

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 761–769, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Discriminative Learning with Natural Annotations: Word Segmentation as a Case Study Wenbin Jiang 1 Meng Sun 1 Yajuan L¨u 1 Yating Yang 2 Qun Liu 3, 1 1Key Laboratory of Intelligent Information Processing Institute of Computing Technology, Chinese Academy of Sciences {jiangwenbin, sunmeng, lvyajuan}@ict.ac.cn 2Multilingual Information Technology Research Center The Xinjiang Technical Institute of Physics & Chemistry, Chinese Academy of Sciences yangyt@ms.xjb.ac.cn 3Centre for Next Generation Localisation Faculty of Engineering and Computing, Dublin City University qliu@computing.dcu.ie Abstract Structural information in web text pro- vides natural annotations for NLP prob- lems such as word segmentation and pars- ing. In this paper we propose a discrim- inative learning algorithm to take advan- tage of the linguistic knowledge in large amounts of natural annotations on the In- ternet. It utilizes the Internet as an external corpus with massive (although slight and sparse) natural annotations, and enables a classifier to evolve on the large-scaled and real-time updated web text. With Chinese word segmentation as a case study, exper- iments show that the segmenter enhanced with the Chinese wikipedia achieves sig- nificant improvement on a series of testing sets from different domains, even with a single classifier and local features. 1 Introduction Problems related to information retrieval, machine translation and social computing need fast and ac- curate text processing, for example, word segmen- tation and parsing. Taking Chinese word seg- mentation for example, the state-of-the-art mod- els (Xue and Shen, 2003; Ng and Low, 2004; Gao et al., 2005; Nakagawa and Uchimoto, 2007; Zhao and Kit, 2008; Jiang et al., 2009; Zhang and Clark, 2010; Sun, 2011b; Li, 2011) are usually trained on human-annotated corpora such as the Penn Chinese Treebank (CTB) (Xue et al., 2005), and perform quite well on corresponding test sets. Since the text used for corpus annotating are usu- ally drawn from specific fields (e.g. newswire or finance), and the annotated corpora are limited in think that NLP has already ... � � � � � � � � � � � � i-1 i j j+1 (a) Natural annotation by hyperlink � � � � � � � � � � � � i-1 i j j+1 � � � � � � � � � � � � i-1 i j j+1 (b) Knowledge for word segmentation (c) Knowledge for dependency parsing Figure 1: Natural annotations for word segmenta- tion and dependency parsing. size (e.g. tens of thousands), the performance of word segmentation tends to degrade sharply when applied to new domains. Internet provides large amounts of raw text, and statistics collected from it have been used to im- prove parsing performance (Nakov and Hearst, 2005; Pitler et al., 2010; Bansal and Klein, 2011; Zhou et al., 2011). The Internet also gives mas- sive (although slight and sparse) natural annota- tions in the forms of structural information includ- ing hyperlinks, fonts, colors and layouts (Sun, …

Content truncated to 500 words