Learning to Recognize Names Across Languages

Learning to Recognize Names Across Languages Anthony F. Gallippi University of Southern California University Park, EEB 234 Los Angeles, CA 90089 USA gallippi @ aludra.usc.edu Abstract The development of natural language pro- ccssing (NLP) systems that perform ma- chine translation (MT) and information re- trieval (IR) has highlighted the need for the automatic recognition of proper names. While various name recognizers have been developed, they suffer from being too lim- ited; some only recognize one name class, and all are language specific. This work develops an approach to multilingual name recognition that allows a system optimized for one language to be ported to another with little additional effort and resources. An initial core set of linguistic features, useful for name recognition in most lan- guages, is identified. When porting to a new language, these features need to be converted (partly by hand, partly by on-line lists), after which point machine learning (ML) techniques build decision trees that map features to name classes. A system initially optimized for English has been successfully ported to Spanish and Japa- nese. Only a few days of human effort for each new language results in performance levels comparable to that of the best cur- rent English systems. 1 Introduction Proper names represent a unique challenge for MT and IR systems. They are not found in dictionaries, are very large in number, come and go every day, and appear in many alias forms. For these reasons, list based matching schemes do not achieve desired performance levels. Hand coded heuristics can be developed to achieve high accuracy, however this approach lacks portability. Much human effort is needed to port the system to a new domain. A desirable approach is one that maximizes reuse and minimizes human effort. This paper presents an approach to proper name recognition that uses ma- chine learning and a language independent fi'ame- work. Knowledge incorporated into the framework is based on a set of measurable linguistic characteris- tics, or ,features. Some of this knowledge is constant across languages. The rest can be generated auto- matically through machine learning techniques. The problem being considered is that of segment- ing natural language text into lexical units, and of tagging those units with various syntactic and se- mantic features. A lexical unit may be a word (e.g., "started") or a phrase (e.g., "The Washington Post"). The particular lexical units of interest here are proper names. Segmenting and tagging proper names is very important for natural language processing, par- ticularly IR and MT. Whether a phrase is a proper name, and what type of proper name it is (company name, location name, person name, date, other) depends on (1) the internal structure of the phrase, and (2) the surrounding con- text. Internal: "Mr. Brandon" Context: "The new company, Safetek, will make air bags." The person title "Mr." reliably shows "Mr. Brandon" to be a person name. "Safetek" can be recognized as a company name by utilizing the preceding contex- tual phrase and appositive "The new company,". The …

Content truncated to 500 words