Word Clustering and Word Selection Based Feature Reduction for MaxEnt Based Hindi NER

Proceedings of ACL-08: HLT, pages 488–495, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Word Clustering and Word Selection based Feature Reduction for MaxEnt based Hindi NER Sujan Kumar Saha Indian Institute of Technology Kharagpur, West Bengal India - 721302 sujan.kr.saha@gmail.com Pabitra Mitra Indian Institute of Technology Kharagpur, West Bengal India - 721302 pabitra@gmail.com Sudeshna Sarkar Indian Institute of Technology Kharagpur, West Bengal India - 721302 shudeshna@gmail.com Abstract Statistical machine learning methods are em- ployed to train a Named Entity Recognizer from annotated data. Methods like Maxi- mum Entropy and Conditional Random Fields make use of features for the training purpose. These methods tend to overfit when the avail- able training corpus is limited especially if the number of features is large or the number of values for a feature is large. To overcome this we proposed two techniques for feature reduction based on word clustering and se- lection. A number of word similarity mea- sures are proposed for clustering words for the Named Entity Recognition task. A few corpus based statistical measures are used for important word selection. The feature reduc- tion techniques lead to a substantial perfor- mance improvement over baseline Maximum Entropy technique. 1 Introduction Named Entity Recognition (NER) involves locat- ing and classifying the names in a text. NER is an important task, having applications in informa- tion extraction, question answering, machine trans- lation and in most other Natural Language Process- ing (NLP) applications. NER systems have been de- veloped for English and few other languages with high accuracy. These belong to two main cate- gories based on machine learning (Bikel et al., 1997; Borthwick, 1999; McCallum and Li, 2003) and lan- guage or domain specific rules (Grishman, 1995; Wakao et al., 1996). In English, the names are usually capitalized which is an important clue for identifying a name. Absence of capitalization makes the Hindi NER task difficult. Also, person names are more diverse in In- dian languages, many common words being used as names. A pioneering work on Hindi NER is by Li and McCallum (2003) where they used Conditional Ran- dom Fields (CRF) and feature induction to auto- matically construct only the features that are impor- tant for recognition. In an effort to reduce overfit- ting, they use a combination of a Gaussian prior and early-stopping. In their Maximum Entropy (MaxEnt) based ap- proach for Hindi NER development, Saha et al. (2008) also observed that the performance of the MaxEnt based model often decreases when huge number of features are used in the model. This is due to overfitting which is a serious problem in most of the NLP tasks in resource poor languages where annotated data is scarce. This paper is a study on effectiveness of word clustering and selection as feature reduction tech- niques for MaxEnt based NER. For clustering we use a number of word similarities like cosine sim- ilarity among words and co-occurrence, along with the k-means clustering algorithm. The clusters are then used as features instead of …

Content truncated to 500 words