InfoXtract location normalization: a hybrid approach to geographic references in information extraction

InfoXtract location normalization: a hybrid approach to geographic references in information extraction ∗∗∗∗ Huifeng Li, Rohini K. Srihari, Cheng Niu, and Wei Li Cymfony Inc. 600 Essjay Road, Williamsville, NY 14221, USA (hli, rohini, cniu, wei)@cymfony.com ∗ This work was partly supported by a grant from the Air Force Research Laboratory’s Information Directorate (AFRL/IF), Rome, NY, under contract F30602-01-C-0035. The authors wish to thank Carrie Pine of AFRL for supporting and commenting this work. Abstract Ambiguity is very high for location names. For example, there are 23 cities named ‘Buffalo’ in the U.S. Based on our previous work, this paper presents a refined hybrid approach to geographic references using our information extraction engine InfoXtract. The InfoXtract location normalization module consists of local pattern matching and discourse co-occurrence analysis as well as default senses. Multiple knowledge sources are used in a number of ways: (i) pattern matching driven by local context, (ii) maximum spanning tree search for discourse analysis, and (iii) applying default sense heuristics and extracting default senses from the web. The results are benchmarked with 96% accuracy on our test collections that consist of both news articles and tourist guides. The performance contribution for each component of the module is also benchmarked and 1 discussed. Introduction The task of location normalization is to decode geographic references for extracted location Named Entities (NE). Ambiguity is a very serious problem for location NEs. For example, there are 23 cities named ‘Buffalo’, including the city in New York State and in the state of Alabama. Country names such as ‘Canada’, ‘Brazil’, and ‘China’ are also city names in the USA. Such ambiguity needs to be properly handled before converting location names into normal form to support Entity Profile (EP) construction, information merging/consolidation as well as visualization of location-stamped extracted events on a map. Location normalization is a special application of word sense disambiguation (WSD). There is considerable research on WSD. Knowledge-based work, such as [Hirst 1987; McRoy 1992; Ng and Lee 1996] used hand-coded rules or supervised machine learning based on an annotated corpus to perform WSD. Recent work emphasizes a corpus-based unsupervised approach [Dagon and Itai 1994; Yarowsky 1992; Yarowsky 1995] that avoids the need for costly truthed training data. Location normalization is different from general WSD in that the selection restriction often used for WSD in many cases is not sufficient to distinguish the correct sense from the other candidates. For example, in the sentence “The White House is located in Washington”, the selection restriction from the collocation ‘located in’ can only determine that “Washington” should be a location name, but is not sufficient to decide the actual sense of this location. In terms of local context, we found that there are certain fairly predictable keyword-driven patterns which can decide the senses of location NEs. These patterns use keywords such as ‘city’, ‘town’, ‘province’, ‘on’, ‘in’ or candidate location subtypes that can be assigned from a location gazetteer. For example, the pattern “X + city” can determine sense tags for …

Content truncated to 500 words