High Precision Extraction of Grammatical Relations

High Precision Extraction of Grammatical Relations John Carroll Cognitive and Computing Sciences University of Sussex Falmer, Brighton BN1 9QH, UK Ted Briscoe Computer Laboratory University of Cambridge JJ Thomson Avenue Cambridge CB3 0FD, UK Abstract A parsing system returning analyses in the form of sets of grammatical relations can obtain high pre- cision if it hypothesises a particular relation only when it is certain that the relation is correct. We operationalise this technique—in a statistical parser using a manually-developed wide-coverage gram- mar of English—by only returning relations that form part of all analyses licensed by the grammar. We observe an increase in precision from 75% to over 90% (at the cost of a reduction in recall) on a test corpus of naturally-occurring text. 1 Introduction1 Head-dependent relationships (possibly labelled with a relation type) have been advocated as a use- ful level of representation for grammatical struc- ture in a number of different large-scale language- processing tasks. For instance, in recent work on statistical treebank grammar parsing (e.g. Collins, 1999) high levels of accuracy have been reached using lexicalised probabilistic models over head- dependent tuples. Bouma, van Noord and Mal- ouf (2001) create dependency treebanks semi-auto- matically in order to induce dependency-based sta- tistical models for parse selection. Lin (1998), Srinivas (2000) and others have evaluated the ac- curacy of both phrase structure-based and depen- dency parsers by matching head-dependent rela- tions against ‘gold standard’ relations, rather than matching (labelled) phrase structure bracketings. Research on unsupervised acquisition of lexical in- formation from corpora, such as argument structure of predicates (Briscoe and Carroll, 1997; McCarthy, 2000), word classes for disambiguation (Clark and Weir, 2001), and collocations (Lin 1999), has used grammatical relation/head/dependent tuples. Such 1A previous version of this paper was presented at IWPT’01; this version contains new experiments and results. tuples also constitute a convenient intermediate rep- resentation in applications such as information ex- traction (Palmer et al., 1993; Yeh, 2000), and docu- ment retrieval on the Web (Grefenstette, 1997). A variety of different approaches have been taken for robust extraction of relation/head/dependent tu- ples, or grammatical relations, from unrestricted text. Dependency parsing is a natural technique to use, and there has been some work in that area on robust analysis and disambiguation (e.g. Laf- ferty, Sleator and Temperley, 1992; Srinivas, 2000). Finite-state approaches (e.g. Karlsson et al., 1995; A¨ıt-Mokhtar and Chanod, 1997; Grefenstette, 1998) have used hand-coded transducers to recognise lin- ear configurations of words and part of speech la- bels associated with, for example, subject/object- verb relationships. An intermediate step may be to mark nominal, verbal etc. ‘chunks’ in the text and to identify the head word of each of the chunks. Sta- tistical finite-state approaches have also been used: Brants, Skut and Krenn (1997) train a cascade of Hidden Markov Models to tag words with their grammatical functions. Approaches based on mem- ory based learning have also used chunking as a first stage, before assigning grammatical relation la- bels to heads of chunks (Argamon, Dagan and Kry- molowski, …

Content truncated to 500 words