Fast Query for Large Treebanks

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 267–275, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Fast Query for Large Treebanks Sumukh Ghodke∗ ∗Department of Computer Science and Software Engineering, University of Melbourne Victoria 3010, Australia Steven Bird∗† †Linguistic Data Consortium, University of Pennsylvania 3600 Market Street, Suite 810 Philadelphia PA 19104, USA Abstract A variety of query systems have been devel- oped for interrogating parsed corpora, or tree- banks. With the arrival of efficient, wide- coverage parsers, it is feasible to create very large databases of trees. However, existing ap- proaches that use in-memory search, or rela- tional or XML database technologies, do not scale up. We describe a method for storage, indexing, and query of treebanks that uses an information retrieval engine. Several experi- ments with a large treebank demonstrate ex- cellent scaling characteristics for a wide range of query types. This work facilitates the cu- ration of much larger treebanks, and enables them to be used effectively in a variety of sci- entific and engineering tasks. 1 Introduction The problem of representing and querying linguistic annotations has been an active area of research for several years. Much of the work has grown from efforts to curate large databases of annotated text such as treebanks, for use in developing and testing language technologies (Marcus et al., 1993; Abeill´e, 2003; Hockenmaier and Steedman, 2007). At least a dozen linguistic tree query languages have been developed for interrogating treebanks (see §2). While high quality syntactic parsers are able to efficiently annotate large quantities of English text (Clark and Curran, 2007), existing approaches to query do not work on the same scale. Many exist- ing systems load the entire corpus into memory and check a user-supplied query against every tree. Oth- ers avoid the memory limitation, and use relational or XML database systems. Although these have built-in support for indexes, they do not scale up ei- ther (Ghodke and Bird, 2008; Zhang et al., 2001)). The ability to interrogate large collections of parsed text has important practical applications. First, it opens the way to a new kind of information retrieval (IR) that is sensitive to syntactic informa- tion, permitting users to do more focussed search. At the simplest level, an ambiguous query term like wind or park could be disambiguated with the help of a POS tag (e.g. wind/N, park/V). (Existing IR engines already support query with part-of-speech tags (Chowdhury and McCabe, 1998)). More com- plex queries could stipulate the syntactic category of apple is in subject position. A second benefit of large scale tree query is for natural language processing. For example, we might compute the likelihood that a given noun appears as the agent or patient of a verb, as a measure of an- imacy. We can use features derived from syntactic trees in order to support semantic role labeling, lan- guage modeling, and information extraction (Chen and Rambow, 2003; Collins et al., 2005; Hakenberg et al., 2009). …

Content truncated to 500 words