A Knowledge-free Method for Capitalized Word Disambiguation

A Knowledge-free Method for Capitalized Word Disambiguation Andrei Mikheev* Harlequin Ltd., Lismore House, 127 George Street, Edinburgh EH72 4JN, UK mikheev@harlequin, co. uk Abstract In this paper we present an approach to the dis- ambiguation of capitalized words when they are used in the positions where capitalization is ex- pected, such as the first word in a sentence or after a period, quotes, etc.. Such words can act as proper names or can be just capitalized vari- ants of common words. The main feature of our approach is that it uses a minimum of pre- built resources and tries to dynamically infer the disambiguation clues from the entire docu- ment. The approach was thoroughly tested and achieved about 98.5% accuracy on unseen texts from The New York Times 1996 corpus. 1 Introduction Disambiguation of capitalized words in mixed- case texts has hardly received much attention in the natural language processing and infor- mation retrieval communities, but in fact it plays an important role in many tasks. Cap- italized words usually denote proper names - names of organizations, locations, people, arti- facts, etc. - but there are also other positions in the text where capitalization is expected. Such ambiguous positions include the first word in a sentence, words in all-capitalized titles or ta- ble entries, a capitalized word after a colon or open quote, the first capitalized word in a list- entry, etc. Capitalized words in these and some other positions present a case of ambiguity - they can stand for proper names as in "White later said ...", or they can be just capitalized common words as in "White elephants are ...". Thus the disambiguation of capitalized words in the ambiguous positions leads to the identifica- tion of proper names I and in this paper we will * Also at HCRC, University of Edinburgh 1This is not entirely true - adjectives derived from lo- cations such as American, French, etc., are always writ- use these two terms interchangeably. Note that this task, does not involve the classification of proper names into semantic categories (person, organization, location, etc.) which is the objec- tive of the Named Entity Recognition task. Many researchers observed that commonly used upper/lower case normalization does not necessarily help document retrieval. Church in (Church, 1995) among other simple text nor- malization techniques studied the effect of case normalization for different words and showed that "...sometimes case variants refer to the same thing (hurricane and Hurricane), some- times they refer to different things (continental and Continental) and sometimes they don't re- fer to much of anything (e.g. anytime and Any- time)." Obviously these differences are due to the fact that some capitalized words stand for proper names (such as Continental- the name of an airline) and some don't. Proper names are the main concern of the Named Entity Recognition subtask (Chinchor, 1998) of Information Extraction. There the dis- ambiguation of the first word of a sentence (and in other ambiguous positions) is one of the cen- tral problems. For …

Content truncated to 500 words