A Synopsis of Learning to Recognize Names Across Languages

A Synopsis of Learning to Recognize Names Across Languages Anthony F. Gallippi University of Southern California University Park, EEB 234 Los Angeles, CA 90089 USA gallippi @ aludra.usc.edu Abstract The development of natural language processing (NLP) systems that perform machine translation (MT) and information retrieval (IR) has highlighted the need for the automatic recognition of proper names. While vari- ous name recognizers have been developed, they suffer from being too limited; some only recognize one name class, and all are language specific. This work devel- ops an approach to multilingual name recognition that uses machine learning and a portable framework to simplify the porting task by maximizing reuse and au- tomation. 1 Introduction Proper names represent a unique challenge for MT and IR systems. They are not found in dictionaries, are very large in number, come and go every day, and ap- pear in many alias forms. For these reasons, list based matching schemes do not achieve desired performance levels. Hand coded heuristics can be developed to achieve high accuracy, however this approach lacks portability. Much human effort is needed to port the system to a new domain. A desirable approach is one that maximizes reuse and minimizes human effort. This paper presents an approach to proper name recognition that uses machine learning and a language independent framework. Knowledge incorporated into the framework is based on a set of measurable linguistic characteristics, or fea- tures. Some of this knowledge is constant across lan- guages. The rest can be generated automatically through machine learning techniques. Whether a phrase (or word) is a proper name, and what type of proper name it is (company name, loca- tion name, person name, date, other) depends on (1) the internal structure of the phrase, and (2) the surrounding context. Internal: 'qVlr. Brandon" Context: 'The new compan.~= Safetek, will make air bags." The person title "Mr." reliably shows "Mr. Brandon" to be a person name. "Safetek" can be recognized as a company name by utilizing the preceding contextual phrase and appositive "The new company,". The recognition task can be broken down into de- limitation and classification. Delimitation is the de- termination of the boundaries of the proper name, while classification serves to provide a more specific category. Original: Delimit: John Smith, chairman of Safetek, announced his resignation yesterday. <PN> John Smith </PN>, chairman of <PN> Safetek </PN> , announced his resignation yesterday. Classify: <person> John Smith </person>, chairman of <company> Safetek </company>, announced his resignation yesterday. During the delimit step, proper name boundaries are identified. Next, the delimited names are categorized. 2 Method The approach taken here is to utilize a data-driven knowledge acquisition strategy based on decision trees which uses contextual information. This differs from other approaches (Farwell et al., 1994; Kitani & Mita- mura, 1994; McDonald, 1993; Rau, 1992) which attempt to achieve this task by: (1) hand-coded heuris- tics, (2) list-based matching schemes, (3) human-gen- erated knowledge bases, and (4) combinations thereof. Delimitation occurs through the application of phrasal templates. These templates, built …

Content truncated to 500 words