A Deterministic Word Dependency Analyzer Enhanced With Preference Learning

A Deterministic Word Dependency Analyzer Enhanced With Preference Learning Hideki Isozaki and Hideto Kazawa and Tsutomu Hirao NTT Communication Science Laboratories NTT Corporation 2-4 Hikaridai, Seikacho, Sourakugun, Kyoto, 619-0237 Japan {isozaki,kazawa,hirao}@cslab.kecl.ntt.co.jp Abstract Word dependency is important in parsing tech- nology. Some applications such as Informa- tion Extraction from biological documents ben- efit from word dependency analysis even with- out phrase labels. Therefore, we expect an ac- curate dependency analyzer trainable without using phrase labels is useful. Although such an English word dependency analyzer was pro- posed by Yamada and Matsumoto, its accu- racy is lower than state-of-the-art phrase struc- ture parsers because of the lack of top-down in- formation given by phrase labels. This paper shows that the dependency analyzer can be im- proved by introducing a Root-Node Finder and a Prepositional-Phrase Attachment Resolver. Experimental results show that these modules based on Preference Learning give better scores than Collins’ Model 3 parser for these subprob- lems. We expect this method is also applicable to phrase structure parsers. 1 Introduction 1.1 Dependency Analysis Word dependency is important in parsing technol- ogy. Figure 1 shows a word dependency tree. Eis- ner (1996) proposed probabilistic models of depen- dency parsing. Collins (1999) used dependency analysis for phrase structure parsing. It is also stud- ied by other researchers (Sleator and Temperley, 1991; Hockenmaier and Steedman, 2002). How- ever, statistical dependency analysis of English sen- tences without phrase labels is not studied very much while phrase structure parsing is intensively studied. Recent studies show that Information Ex- traction (IE) and Question Answering (QA) benefit from word dependency analysis without phrase la- bels. (Suzuki et al., 2003; Sudo et al., 2003) Recently, Yamada and Matsumoto (2003) pro- posed a trainable English word dependency ana- lyzer based on Support Vector Machines (SVM). They did not use phrase labels by considering an- notation of documents in expert domains. SVM (Vapnik, 1995) has shown good performance in dif- He a girl a telescope with saw He saw a girl with a telescope. Figure 1: A word dependency tree ferent tasks of Natural Language Processing (Kudo and Matsumoto, 2001; Isozaki and Kazawa, 2002). Most machine learning methods do not work well when the number of given features (dimensionality) is large, but SVM is relatively robust. In Natural Language Processing, we use tens of thousands of words as features. Therefore, SVM often gives good performance. However, the accuracy of Yamada’s analyzer is lower than state-of-the-art phrase structure parsers such as Charniak’s Maximum-Entropy-Inspired Parser (MEIP) (Charniak, 2000) and Collins’ Model 3 parser. One reason is the lack of top-down infor- mation that is available in phrase structure parsers. In this paper, we show that the accuracy of the word dependency parser can be improved by adding a base-NP chunker, a Root-Node Finder, and a Prepositional Phrase (PP) Attachment Resolver. We introduce the base-NP chunker because base NPs are important components of a sentence and can be easily annotated. Since most words are contained in a base NP or are adjacent to a …

Content truncated to 500 words