Parsing with Compositional Vector Grammars

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 455–465, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Parsing with Compositional Vector Grammars Richard Socher John Bauer Christopher D. Manning Andrew Y. Ng Computer Science Department, Stanford University, Stanford, CA 94305, USA richard@socher.org, horatio@gmail.com, manning@stanford.edu, ang@cs.stanford.edu Abstract Natural language parsing has typically been done with small sets of discrete cat- egories such as NP and VP, but this rep- resentation does not capture the full syn- tactic nor semantic richness of linguistic phrases, and attempts to improve on this by lexicalizing phrases or splitting cate- gories only partly address the problem at the cost of huge feature spaces and sparse- ness. Instead, we introduce a Compo- sitional Vector Grammar (CVG), which combines PCFGs with a syntactically un- tied recursive neural network that learns syntactico-semantic, compositional vector representations. The CVG improves the PCFG of the Stanford Parser by 3.8% to obtain an F1 score of 90.4%. It is fast to train and implemented approximately as an efficient reranker it is about 20% faster than the current Stanford factored parser. The CVG learns a soft notion of head words and improves performance on the types of ambiguities that require semantic information such as PP attachments. 1 Introduction Syntactic parsing is a central task in natural lan- guage processing because of its importance in me- diating between linguistic expression and mean- ing. For example, much work has shown the use- fulness of syntactic representations for subsequent tasks such as relation extraction, semantic role la- beling (Gildea and Palmer, 2002) and paraphrase detection (Callison-Burch, 2008). Syntactic descriptions standardly use coarse discrete categories such as NP for noun phrases or PP for prepositional phrases. However, recent work has shown that parsing results can be greatly improved by defining more fine-grained syntactic (riding,V, ) (a,Det, ) (bike,NN, ) (a bike,NP, ) (riding a bike,VP, ) Discrete Syntactic – Continuous Semantic Representations in the Compositional Vector Grammar Figure 1: Example of a CVG tree with (cate- gory,vector) representations at each node. The vectors for nonterminals are computed via a new type of recursive neural network which is condi- tioned on syntactic categories from a PCFG. categories, which better capture phrases with simi- lar behavior, whether through manual feature engi- neering (Klein and Manning, 2003a) or automatic learning (Petrov et al., 2006). However, subdi- viding a category like NP into 30 or 60 subcate- gories can only provide a very limited represen- tation of phrase meaning and semantic similarity. Two strands of work therefore attempt to go fur- ther. First, recent work in discriminative parsing has shown gains from careful engineering of fea- tures (Taskar et al., 2004; Finkel et al., 2008). Fea- tures in such parsers can be seen as defining effec- tive dimensions of similarity between categories. Second, lexicalized parsers (Collins, 2003; Char- niak, 2000) associate each category with a lexical item. This gives a fine-grained notion of semantic similarity, which is useful for tackling problems like ambiguous attachment decisions. …

Content truncated to 500 words