An Empirical Evaluation of LFG-DOP

An Empirical Evaluation of LFG-DOP Rens Bod Informatics Research Institute, University of Leeds, Leeds LS2 9JT, & Institute for Logic, Language and Computation, University of Amsterdam rens@scs.leeds.ac.uk Abstract This paper presents an empirical assessment of the LFG- DOP model introduced by Bod & Kaplan (1998). The parser we describe uses fragments from LFG-annotated sentences to parse new sentences and Monte Carlo techniques to compute the most probable parse. While our main goal is to test Bod & Kaplan's model, we will also test a version of LFG-DOP which treats generalized fragments as previously unseen events. Experiments with the Verbmobil and Homecentre corpora show that our version of LFG-DOP outperforms Bod & Kaplan's model, and that LFG's functional information improves the parse accuracy of tree structures. 1 Introduction We present an empirical evaluation of the LFG-DOP model introduced by Bod & Kaplan (1998). LFG-DOP is a Data-Oriented Parsing (DOP) model (Bod 1993, 98) based on the syntactic representations of Lexical- Functional Grammar (Kaplan & Bresnan 1982). A DOP model provides linguistic representations for an unlimit- ed set of sentences by generalizing from a given corpus of annotated exemplars. It operates by decomposing the given representations into (arbitrarily large) fragments and recomposing those pieces to analyze new sentences. The occurrence-frequencies of the fragments are used to determine the most probable analysis of a sentence. So far, DOP models have been implemented for phrase-structure trees and logical-semantic represent- ations (cf. Bod 1993, 98; Sima'an 1995, 99; Bonnema et al. 1997; Goodman 1998). However, these DOP models are limited in that they cannot account for underlying syntactic and semantic dependencies that are not reflected directly in a surface tree. DOP models for a number of richer representations have been explored (van den Berg et al. 1994; Tugwell 1995), but these approaches have remained context-free in their generative power. In contrast, Lexical-Functional Grammar (Kaplan & Bresnan 1982) is known to be beyond context-free. In Bod & Kaplan (1998), a first DOP model was proposed based on representations defined by LFG theory ("LFG-DOP").1 This model was 1 DOP models have recently also been proposed for Tree- Adjoining Grammar and Head-driven Phrase Structure Grammar (cf. Neumann & Flickinger 1999). studied from a mathematical perspective by Cormons (1999) who also accomplished a first simple experiment with LFG-DOP. Next, Way (1999) studied LFG-DOP as an architecture for machine translation. The current paper contains the first extensive empirical evaluation of LFG-DOP on the currently available LFG-annotated corpora: the Verbmobil corpus and the Homecentre corpus. Both corpora were annotated at Xerox PARC. Our parser uses fragments from LFG-annotated sentences to parse new sentences, and Monte Carlo techniques to compute the most probable parse. Although our main goal is to test Bod & Kaplan's LFG- DOP model, we will also test a modified version of LFG-DOP which uses a different model for computing fragment probabilities. While Bod & Kaplan treat all fragments probabilistically equal regardless whether they contain generalized features, we will propose a more fine-grained probability model which treats fragments with …

Content truncated to 500 words