Towards Efficient Named-Entity Rule Induction for Customizability

Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 128–138, Jeju Island, Korea, 12–14 July 2012. c⃝2012 Association for Computational Linguistics Towards Efficient Named-Entity Rule Induction for Customizability Ajay Nagesh1,2 1IITB-Monash Research Academy ajaynagesh@cse.iitb.ac.in Ganesh Ramakrishnan 2IIT Bombay ganesh@cse.iitb.ac.in Laura Chiticariu IBM Research - Almaden chiti@us.ibm.com Rajasekar Krishnamurthy IBM Research - Almaden rajase@us.ibm.com Ankush Dharkar SASTRA University ankushdharkar@cse.sastra.edu Pushpak Bhattacharyya IIT Bombay pb@cse.iitb.ac.in Abstract Generic rule-based systems for Information Extraction (IE) have been shown to work reasonably well out-of-the-box, and achieve state-of-the-art accuracy with further domain customization. However, it is generally rec- ognized that manually building and customiz- ing rules is a complex and labor intensive pro- cess. In this paper, we discuss an approach that facilitates the process of building cus- tomizable rules for Named-Entity Recognition (NER) tasks via rule induction, in the Annota- tion Query Language (AQL). Given a set of basic features and an annotated document col- lection, our goal is to generate an initial set of rules with reasonable accuracy, that are in- terpretable and thus can be easily refined by a human developer. We present an efficient rule induction process, modeled on a four- stage manual rule development process and present initial promising results with our sys- tem. We also propose a simple notion of ex- tractor complexity as a first step to quantify the interpretability of an extractor, and study the effect of induction bias and customization of basic features on the accuracy and complex- ity of induced rules. We demonstrate through experiments that the induced rules have good accuracy and low complexity according to our complexity measure. 1 Introduction Named-entity recognition (NER) is the task of iden- tifying mentions of rigid designators from text be- longing to named-entity types such as persons, orga- nizations and locations (Nadeau and Sekine, 2007). Generic NER rules have been shown to work reason- ably well-out-of-the-box, and with further domain customization (Chiticariu et al., 2010b), achieve quality surpassing state-of-the-art results. Table 1 System Dataset Fβ=1 Generic Customized GATE ACE2002 57.8 82.2 ACE 2005 57.32 88.95 SystemT CoNLL 2003 64.15 91.77 Enron 76.53 85.29 Table 1: Quality of generic vs. customized rules. summarizes the quality of NER rules out-of-the-box and after domain customization in the GATE (Cun- ningham et al., 2011) and SystemT (Chiticariu et al., 2010a) systems, as reported in (Maynard et al., 2003) and (Chiticariu et al., 2010b) respectively. Rule-based systems are widely used in enterprise settings due to their explainability. Rules are trans- parent, which leads to better explainability of errors. One can easily identify the cause of a false positive or negative, and refine the rules without affecting other correct results identified by the system. Fur- thermore, rules are typically easier to understand by an IE developer and can be customized for a new domain without requiring additional labeled data. Typically, a rule-based NER system consists of a combination of four categories of rules (Chiticariu et al., 2010b): (1) Basic Feature (BF) rules to identify …

Content truncated to 500 words