A Latent Variable Model of Synchronous Parsing for Syntactic and Semantic Dependencies

CoNLL 2008: Proceedings of the 12th Conference on Computational Natural Language Learning, pages 178–182 Manchester, August 2008 A Latent Variable Model of Synchronous Parsing for Syntactic and Semantic Dependencies James Henderson Dept Computer Science Univ Geneva james.henderson@ cui.unige.ch Paola Merlo Dept Linguistics Univ Geneva merlo@ lettres.unige.ch Gabriele Musillo Depts Linguistics and Computer Science Univ Geneva musillo@ lettres.unige.ch Ivan Titov∗ Dept Computer Science Univ Illinois at U-C titov@uiuc.edu Abstract We propose a solution to the challenge of the CoNLL 2008 shared task that uses a generative history-based latent variable model to predict the most likely derivation of a synchronous dependency parser for both syntactic and semantic dependencies. The submitted model yields 79.1% macro- average F1 performance, for the joint task, 86.9% syntactic dependencies LAS and 71.0% semantic dependencies F1. A larger model trained after the deadline achieves 80.5% macro-average F1, 87.6% syntac- tic dependencies LAS, and 73.1% seman- tic dependencies F1. 1 Introduction Successes in syntactic tasks, such as statistical parsing and tagging, have recently paved the way to statistical learning techniques for levels of se- mantic representation, such as recovering the log- ical form of a sentence for information extraction and question-answering applications (e.g. (Wong and Mooney, 2007)) or jointly learning the syntac- tic structure of the sentence and the propositional argument-structure of its main predicates (Musillo and Merlo, 2006; Merlo and Musillo, 2008). In this vein, the CoNLL 2008 shared task sets the challenge of learning jointly both syntactic depen- dencies (extracted from the Penn Treebank (Mar- cus et al., 1993) ) and semantic dependencies (ex- tracted both from PropBank (Palmer et al., 2005) ∗c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. 0Authors in alphabetical order. and NomBank (Meyers et al., 2004) under a uni- fied representation. We propose a solution that uses a generative history-based model to predict the most likely derivation of a synchronous dependency parser for both syntactic and semantic dependencies. Our probabilistic model is based on Incremental Sig- moid Belief Networks (ISBNs), a recently pro- posed latent variable model for syntactic struc- ture prediction, which has shown very good be- haviour for both constituency (Titov and Hender- son, 2007a) and dependency parsing (Titov and Henderson, 2007b). The ability of ISBNs to in- duce their features automatically enables us to ex- tend this architecture to learning a synchronous parse of syntax and semantics without modifica- tion of the main architecture. By solving the problem with synchronous parsing, a probabilistic model is learnt which maximises the joint proba- bility of the syntactic and semantic dependencies and thereby guarantees that the output structure is globally coherent, while at the same time building the two structures separately. This extension of the ISBN architecture is therefore applicable to other problems where two independent, but related, lev- els of representation are being learnt, such as sta- tistical machine translation. Currently the largest model we have trained achieves 80.5% macro-average F1 performance for the joint task, 87.6% syntactic dependencies LAS, and 73.1% semantic …

Content truncated to 500 words