Trained Named Entity Recognition using Distributional Clusters

Trained Named Entity Recognition Using Distributional Clusters Dayne Freitag HNC Software, LLC 3661 Valley Centre Drive San Diego, CA 92130 DayneFreitag@fairisaac.com Abstract This work applies boosted wrapper induction (BWI), a machine learning algorithm for informa- tion extraction from semi-structured documents, to the problem of named entity recognition. The de- fault feature set of BWI is augmented with features based on distributional term clusters induced from a large unlabeled text corpus. Using no traditional lin- guistic resources, such as syntactic tags or special- purpose gazetteers, this approach yields results near the state of the art in the MUC 6 named entity do- main. Supervised learning using features derived through unsupervised corpus analysis may be re- garded as an alternative to bootstrapping methods. 1 Introduction The problem of named entity recognition (NER) has recently received increasing attention. Identifica- tion of generic semantic categories in text—such as mentions of people, organizations, locations, and temporal and numeric expressions—is a necessary first step in many applications of information ex- traction, information retrieval, and question answer- ing. To a large extent, knowledge-poor methods suf- fice to yield good recognition performance. In par- ticular, supervised learning can be used to produce a system with performance at or near the state of the art (Bikel et al., 1997). In the supervised learning framework, a corpus of (typically) a few hundred documents is annotated by hand to identify the entities of interest. Features of local context are then used to train a system to distinguish instances from non-instances in novel texts. Such features may include literal word tests, patterns of orthography, parts of speech, seman- tic categories, or membership in special-purpose gazetteers. While supervised training greatly facilitates the development of a robust NER system, the re- quirement of a substantial training corpus remains an impediment to the rapid deployment of NER in new domains or new languages. A number bush peters reagan noriega ... john robert james david ... president chairman head owner ... japan california london chicago ... Table 1: Sample members of four clusters from the Wall Street Journal corpus. of researchers have therefore sought to exploit the availability of unlabeled documents, typically by bootstrapping a classifier using automatic la- bellings (Collins and Singer, 1999; Cucerzan and Yarowsky, 1999; Thelen and Riloff, 2002). Here, we investigate a different approach. Us- ing a distributional clustering technique called co- clustering, we produce clusters which, intuitively, should be useful for NER. Table 1 shows exam- ple terms from several sample clusters induced us- ing a collection of documents from the Wall Street Journal (WSJ). Several papers have shown that dis- tributional clustering yields categories that have high agreement with part of speech (Sch¨utze, 1995; Clark, 2000). As the table illustrates, these clus- ters also tend to have a useful semantic dimension. Clustering on the WSJ portion of the North Ameri- can News corpus yields two clusters that clearly cor- respond to personal names, one for first names and one for last names. As an experiment, we scanned the MUC6 NER …

Content truncated to 500 words