Domain Adaptation of Rule-Based Annotators for Named-Entity Recognition Tasks

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1002–1012, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Domain Adaptation of Rule-Based Annotators for Named-Entity Recognition Tasks Laura Chiticariu Rajasekar Krishnamurthy Yunyao Li Frederick Reiss Shivakumar Vaithyanathan IBM Research – Almaden 650 Harry Road, San Jose, CA 95120, USA {chiti, rajase, yunyaoli, frreiss}@us.ibm.com, shiv@almaden.ibm.com Abstract Named-entity recognition (NER) is an impor- tant task required in a wide variety of ap- plications. While rule-based systems are ap- pealing due to their well-known “explainabil- ity,” most, if not all, state-of-the-art results for NER tasks are based on machine learning techniques. Motivated by these results, we ex- plore the following natural question in this pa- per: Are rule-based systems still a viable ap- proach to named-entity recognition? Specif- ically, we have designed and implemented a high-level language NERL on top of Sys- temT, a general-purpose algebraic informa- tion extraction system. NERL is tuned to the needs of NER tasks and simplifies the pro- cess of building, understanding, and customiz- ing complex rule-based named-entity annota- tors. We show that these customized annota- tors match or outperform the best published results achieved with machine learning tech- niques. These results confirm that we can reap the benefits of rule-based extractors’ ex- plainability without sacrificing accuracy. We conclude by discussing lessons learned while building and customizing complex rule-based annotators and outlining several research di- rections towards facilitating rule development. 1 Introduction Named-entity recognition (NER) is the task of iden- tifying mentions of rigid designators from text be- longing to named-entity types such as persons, orga- nizations and locations (Nadeau and Sekine, 2007). While NER over formal text such as news articles and webpages is a well-studied problem (Bikel et al., 1999; McCallum and Li, 2003; Etzioni et al., 2005), there has been recent work on NER over in- formal text such as emails and blogs (Huang et al., 2001; Poibeau and Kosseim, 2001; Jansche and Ab- ney, 2002; Minkov et al., 2005; Gruhl et al., 2009). The techniques proposed in the literature fall under three categories: rule-based (Krupka and Hausman, 2001; Sekine and Nobata, 2004), machine learning- based (O. Bender and Ney, 2003; Florian et al., 2003; McCallum and Li, 2003; Finkel and Manning, 2009; Singh et al., 2010) and hybrid solutions (Sri- hari et al., 2001; Jansche and Abney, 2002). 1.1 Motivation Although there are well-established rule-based sys- tems to perform NER tasks, most, if not all, state-of- the-art results for NER tasks are based on machine learning techniques. However, the rule-based ap- proach is still extremely appealing due to the associ- ated transparency of the internal system state, which leads to better explainability of errors (Siniakov, 2010). Ideally, one would like to benefit from the transparency and explainability of rule-based tech- niques, while achieving state-of-the-art accuracy. A particularly challenging aspect of rule-based NER in practice is domain customization — cus- tomizing existing annotators to produce accurate re- sults in new domains. In machine learning-based systems, adapting to a …

Content truncated to 500 words