Generalizing a Strongly Lexicalized Parser using Unlabeled Data

Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 126–134, Gothenburg, Sweden, April 26-30 2014. c⃝2014 Association for Computational Linguistics Generalizing a Strongly Lexicalized Parser using Unlabeled Data Tejaswini Deoskar1, Christos Christodoulopoulos2, Alexandra Birch1, Mark Steedman1 1School of Informatics, University of Edinburgh, Edinburgh, EH8 9AB 2University of Illinois, Urbana-Champaign, Urbana, IL 61801 {tdeoskar,abmayne,steedman}@inf.ed.ac.uk, christod@illinois.edu Abstract Statistical parsers trained on labeled data suffer from sparsity, both grammatical and lexical. For parsers based on strongly lexicalized grammar formalisms (such as CCG, which has complex lexical cate- gories but simple combinatory rules), the problem of sparsity can be isolated to the lexicon. In this paper, we show that semi-supervised Viterbi-EM can be used to extend the lexicon of a generative CCG parser. By learning complex lexical entries for low-frequency and unseen words from unlabeled data, we obtain improvements over our supervised model for both in- domain (WSJ) and out-of-domain (ques- tions and Wikipedia) data. Our learnt lexicons when used with a discriminative parser such as C&C also significantly im- prove its performance on unseen words. 1 Introduction An important open problem in natural language parsing is to generalize supervised parsers, which are trained on hand-labeled data, using unlabeled data. The problem arises because further hand- labeled data in the amounts necessary to signif- icantly improve supervised parsers are very un- likely to be made available. Generalization is also necessary in order to achieve good performance on parsing in textual domains other than the domain of the available labeled data. For example, parsers trained on Wall Street Journal (WSJ) data suffer a fall in accuracy on other domains (Gildea, 2001). In this paper, we use self-training to generalize the lexicon of a Combinatory Categorial Gram- mar (CCG) (Steedman, 2000) parser. CCG is a strongly lexicalized formalism, in which every word is associated with a syntactic category (sim- ilar to an elementary syntactic structure) indicat- ing its subcategorization potential. Lexical en- tries are fine-grained and expressive, and contain a large amount of language-specific grammatical information. For parsers based on strongly lexical- ized formalisms, the problem of grammar general- ization can be cast largely as a problem of lexical extension. The present paper focuses on learning lexi- cal categories for words that are unseen or low- frequency in labeled data, from unlabeled data. Since lexical categories in a strongly lexicalized formalism are complex, fine-grained (and far more numerous than simple part-of-speech tags), they are relatively sparse in labeled data. Despite per- forming at state-of-the-art levels, a major source of error made by CCG parsers is related to unseen and low-frequency words (Hockenmaier, 2003; Clark and Curran, 2007; Thomforde and Steed- man, 2011). The unseen words for which we learn categories are surprisingly commonplace words of English; examples are conquered, apprehended, subdivided, scoring, denotes, hunted, obsessed, residing, migrated (Wikipedia). Correctly learn- ing to parse the predicate-argument structures as- sociated with such words (expressed as lexical cat- egories in the case of CCG), is important for open- domain parsing, not only …

Content truncated to 500 words