Japanese Named Entity Extraction with Redundant Morphological Analysis

Japanese Named Entity Extraction with Redundant Morphological Analysis Masayuki Asahara and Yuji Matsumoto Graduate School of Information Science Nara Institute of Science and Technology, Japan � masayu-a,matsu � @is.aist-nara.ac.jp Abstract Named Entity (NE) extraction is an important subtask of document processing such as in- formation extraction and question answering. A typical method used for NE extraction of Japanese texts is a cascade of morphological analysis, POS tagging and chunking. However, there are some cases where segmentation gran- ularity contradicts the results of morphologi- cal analysis and the building units of NEs, so that extraction of some NEs are inherently im- possible in this setting. To cope with the unit problem, we propose a character-based chunk- ing method. Firstly, the input sentence is an- alyzed redundantly by a statistical morpholog- ical analyzer to produce multiple (n-best) an- swers. Then, each character is annotated with its character types and its possible POS tags of the top n-best answers. Finally, a support vec- tor machine-based chunker picks up some por- tions of the input sentence as NEs. This method introduces richer information to the chunker than previous methods that base on a single morphological analysis result. We apply our method to IREX NE extraction task. The cross validation result of the F-measure being 87.2 shows the superiority and effectiveness of the method. 1 Introduction Named Entity (NE) extraction aims at identifying proper nouns and numerical expressions in a text, such as per- sons, locations, organizations, dates, and so on. This is an important subtask of document processing like infor- mation extraction and question answering. A common standard data set for Japanese NE extrac- tion is provided by IREX workshop (IREX Committee, editor, 1999). Generally, Japanese NE extraction is done in the following steps: Firstly, a Japanese text is seg- mented into words and is annotated with POS tags by a morphological analyzer. Then, a chunker brings together the words into NE chunks based on contextual informa- tion. However, such a straightforward method cannot ex- tract NEs whose segmentation boundary contradicts that of morphological analysis outputs. For example, a sen- tence “ ���� � � � ” is segmented as “ �� / ��� / � /  /  /  / � ” by a morphological analyzer. “ ��� �� ” (“Koizumi Jun’ichiro” – family and first names) as a person name and “   ” (“Septem- ber”) as a date will be extracted by combining word units. On the other hand, “  ” (abbreviation of North Korea) cannot be extracted as a name of location because it is contained by the word unit “ � ” (visiting North Korea). Figure 1 illustrates the example with English translation. Some previous works try to cope with the word unit problem: Uchimoto (Uchimoto et al., 2000) introduces transformation rules to modify the word units given by a morphological analyzer. Isozaki (Isozaki and Kazawa, 2002) controls the parameters of a statistical morpholog- ical analyzer so as to produce more fine-grained output. These method …

Content truncated to 500 words