Augmented Parsing of Unknown Word by Graph-Based Semi-Supervised Learning

Augmented Parsing of Unknown Word by Graph-based Semi-supervised Learning Qiuping Huang Derek F. Wong Lidia S. Chao Xiaodong Zeng Liangye He NLP2CT Laboratory / Department of Computer and Information Science University of Macau Macau S.A.R., China michellehuang718@gmail.com,{derekfw,lidiasc}@umac.mo, nlp2ct.samuel@gmail.com,wutianshui0515@gmail.com Abstract This paper presents a novel method using graph-based semi-supervised learning (SSL) to improve the syntax parsing of unknown words. Different from conventional approaches that uses hand-crafted rules, rich morphological features, or a character-based model to handle unknown words, this method is based on a graph-based label propagation technique. It gives greater improvement on grammars trained on a smaller amount of labeled data and a large amount of unlabeled one. A transductiv1 graph-based SSL method is employed to propagate POS and derive the emission distributions from labeled data to unlabeled one. The derived distributions are incorporated into the parsing process. The proposed method effectively augments the original supervised parsing model by contributing 2.28% and 1.72% absolute improvement on the accuracy of POS tagging and syntax parsing for Penn Chinese Treebank respectively. 1 Introduction Parsing is an important and fundamental task in natural language processing. In the past years, many researches focusing on building high quality parsers for English (Charniak, 2000; Collins, 2003; Charniak and Johnson, 2005; Petrov et al., 2006) and these parsers obtain the state-of-the-art performance up to 92% accuracy. 1Transductive learning is used to contrast inductive learning. A learner is transductive if it only works on the labeled and unlabeled training data, and cannot handle unseen data. Recently, Chinese parsing has received more and more attention, and several researchers attempt to develop accurate parsers for Chinese (Klein and Manning, 2003; Charniak and Johnson, 2005; Petrov and Klein, 2007). Inspired from their works, Huang et al., (2012) design a head propagation table to improve the parsing performance with a factored model. Nevertheless, as pointed out in (Harper and Huang, 2009), the improved performance around 84% F-measure that still falls far short of performance on English. This leaves a large space for the further improvement of Chinese parsing. As far as we known, there is a large portion of fixed errors stemming from unknown words in Chinese parsing. Therefore, a robust parser must have a mechanism of processing unknown words, where it discovers the POS tag and features information about unknown words during parsing. A number of researches design hand- crafted rules or make use of rich morphological features to handle them. It is well known that Chinese words tend to have greater POS tag ambiguities than English and the morphological properties of Chinese words are complicated to be predicted of POS type for unknown words. For this reason, Harper and Huang (2007) present a character-based model to handle Chinese unknown words. Similar to their work, He et al., (2012) propose a more effective method. They mainly use an exponential function to represent the distance between the head character and other characters in an unknown word and use the geometric average to estimate the emission probability of it. However, in this paper, …

Content truncated to 500 words