Transfer Learning for Constituency-Based Grammars

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 291–301, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Transfer Learning for Constituency-Based Grammars Yuan Zhang, Regina Barzilay Massachusetts Institute of Technology {yuanzh, regina}@csail.mit.edu Amir Globerson The Hebrew University gamir@cs.huji.ac.il Abstract In this paper, we consider the problem of cross-formalism transfer in parsing. We are interested in parsing constituency- based grammars such as HPSG and CCG using a small amount of data specific for the target formalism, and a large quan- tity of coarse CFG annotations from the Penn Treebank. While all of the target formalisms share a similar basic syntactic structure with Penn Treebank CFG, they also encode additional constraints and se- mantic features. To handle this appar- ent discrepancy, we design a probabilistic model that jointly generates CFG and tar- get formalism parses. The model includes features of both parses, allowing trans- fer between the formalisms, while pre- serving parsing efficiency. We evaluate our approach on three constituency-based grammars — CCG, HPSG, and LFG, aug- mented with the Penn Treebank-1. Our ex- periments show that across all three for- malisms, the target parsers significantly benefit from the coarse annotations.1 1 Introduction Over the last several decades, linguists have in- troduced many different grammars for describing the syntax of natural languages. Moreover, the ongoing process of developing new formalisms is intrinsic to linguistic research. However, before these grammars can be used for statistical pars- ing, they require annotated sentences for training. The difficulty of obtaining such annotations is a key limiting factor that inhibits the effective use of these grammars. 1The source code for the work is available at http://groups.csail.mit.edu/rbg/code/ grammar/acl2013. The standard solution to this bottleneck has re- lied on manually crafted transformation rules that map readily available syntactic annotations (e.g, the Penn Treebank) to the desired formalism. De- signing these transformation rules is a major un- dertaking which requires multiple correction cy- cles and a deep understanding of the underlying grammar formalisms. In addition, designing these rules frequently requires external resources such as Wordnet, and even involves correction of the existing treebank. This effort has to be repeated for each new grammar formalism, each new anno- tation scheme and each new language. In this paper, we propose an alternative ap- proach for parsing constituency-based grammars. Instead of using manually-crafted transformation rules, this approach relies on a small amount of annotations in the target formalism. Frequently, such annotations are available in linguistic texts that introduce the formalism. For instance, a textbook on HPSG (Pollard and Sag, 1994) il- lustrates grammatical constructions using about 600 examples. While these examples are infor- mative, they are not sufficient for training. To compensate for the annotation sparsity, our ap- proach utilizes coarsely annotated data readily available in large quantities. A natural candidate for such coarse annotations is context-free gram- mar (CFG) from the Penn Treebank, while the target formalism can be any constituency-based grammars, such as Combinatory Categorial Gram- mar (CCG) (Steedman, 2001), Lexical Functional Grammar (LFG) …

Content truncated to 500 words