Named Entity Recognition Using a Character-based Probabilistic Approach

Named Entity Recognition Using a Character-based Probabilistic Approach Casey Whitelaw and Jon Patrick Language Technology Research Group Capital Markets Co-operative Research Centre University of Sydney {casey,jonpat}@it.usyd.edu.au Abstract We present a named entity recognition and classification system that uses only probabilis- tic character-level features. Classifications by multiple orthographic tries are combined in a hidden Markov model framework to incorpo- rate both internal and contextual evidence. As part of the system, we perform a preprocess- ing stage in which capitalisation is restored to sentence-initial and all-caps words with high accuracy. We report f-values of 86.65 and 79.78 for English, and 50.62 and 54.43 for the German datasets. 1 Introduction Language independent NER requires the development of a metalinguistic model that is sufficiently broad to ac- commodate all languages, yet can be trained to exploit the specific features of the target language. Our aim in this paper is to investigate the combination of a character- level model, orthographic tries, with a sentence-level hid- den Markov model. The local model uses affix informa- tion from a word and its surrounds to classify each word independently, and relies on the sentence-level model to determine a correct state sequence. Capitalisation is an often-used discriminator for NER, but can be misleading in sentence-initial or all-caps text. We choose to use a model that makes no assumptions about the capitalisation scheme, or indeed the character set, of the target language. We solve the problem of mis- leading case in a novel way by removing the effects of sentence-initial or all-caps capitalisation. This results in a simpler language model and easier recognition of named entities while remaining strongly language independent. 2 Probabilistic Classification using Orthographic Tries Tries are an efficient data structure for capturing statis- tical differences between strings in different categories. In an orthographic trie, a path from the root through n nodes represents a string a1a2 . . . an. The n-th node in the path stores the occurrences (frequency) of the string a1a2 . . . an in each word category. These fre- quencies can be used to calculate probability estimates P(c | a1a2 . . . an) for each category c. Tries have previ- ously been used in both supervised (Patrick et al., 2002) and unsupervised (Cucerzan and Yarowsky, 1999) named entity recognition. Each node in an orthographic trie stores the cumula- tive frequency information for each category in which a given string of characters occurs. A heterogeneous node represents a string that occurs in more than one category, while a homogeneous node represents a string that occurs in only one category. If a string a1a2 . . . an occurs in only one category, all longer strings a1a2 . . . an . . . an+k are also of the same category. This redundancy can be exploited when constructing a trie. We build minimum- depth MD-tries which have the condition that all nodes are heterogeneous, and all leaves are homogeneous. MD- tries are only as large as is necessary to capture the dif- ferences …

Content truncated to 500 words