Rule-Based Information Extraction is Dead! Long Live Rule-Based Information Extraction Systems!

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 827–832, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Rule-based Information Extraction is Dead! Long Live Rule-based Information Extraction Systems! Laura Chiticariu IBM Research - Almaden San Jose, CA chiti@us.ibm.com Yunyao Li IBM Research - Almaden San Jose, CA yunyaoli@us.ibm.com Frederick R. Reiss IBM Research - Almaden San Jose, CA frreiss@us.ibm.com Abstract The rise of “Big Data” analytics over unstruc- tured text has led to renewed interest in infor- mation extraction (IE). We surveyed the land- scape of IE technologies and identified a major disconnect between industry and academia: while rule-based IE dominates the commercial world, it is widely regarded as dead-end tech- nology by the academia. We believe the dis- connect stems from the way in which the two communities measure the benefits and costs of IE, as well as academia’s perception that rule- based IE is devoid of research challenges. We make a case for the importance of rule-based IE to industry practitioners. We then lay out a research agenda in advancing the state-of-the- art in rule-based IE systems which we believe has the potential to bridge the gap between academic research and industry practice. 1 Introduction The recent growth of “Big Data” analytics over large quantities of unstructured text has led to increased interest in information extraction technologies from both academia and industry (Mendel, 2013). Most recent academic research in this area starts from the assumption that statistical machine learn- ing is the best approach to solving information ex- traction problems. Figure 1 shows empirical ev- idence of this trend drawn from a survey of re- cent published research papers. We examined the EMNLP, ACL, and NAACL conference proceedings from 2003 through 2012 and identified 177 different EMNLP research papers on the topic of entity ex- traction. We then classified these papers into three categories, based on the techniques used: purely Commercial*Vendors*(2013)* NLP*Papers* (200392012)* 100%$ 50%$ 0%$ 3.5%* 21%$ 75%$ Rule,$ Based$ Hybrid$ Machine$ Learning$ Based$ 45%* 22%$ 33%$ Implementa@ons*of*En@ty*Extrac@on* Large*Vendors* 67%* 17%$ 17%$ All*Vendors* Figure 1: Fraction of NLP conference papers from EMNLP, ACL, and NAACL over 10 years that use ma- chine learning versus rule-based techniques to perform entity extraction over text (left); the same breakdown for commercial entity extraction vendors one year after the end of this 10-year period (right). The rule-based ap- proach, although largely ignored in the research commu- nity, dominates the commercial market. rule-based, purely machine learning-based, or a hy- brid of the two. We focus on entity extraction, as it is a classical IE task, and most industrial IE systems offer this feature. The left side of the graph shows the breakdown of research papers according to this categorization. Only six papers relied solely on rules to perform the extraction tasks described. The remainder relied en- tirely or substantially on statistical techniques. As shown in Figure 2, these fractions were roughly con- stant across the 10-year period studied, indicating that attitudes regarding the relative …

Content truncated to 500 words