Closing the Gap: Learning-Based Information Extraction Rivaling Knowledge-Engineering Methods

Closing the Gap: Learning-Based Information Extraction Rivaling Knowledge-Engineering Methods Hai Leong Chieu DSO National Laboratories 20 Science Park Drive Singapore 118230 chaileon@dso.org.sg Hwee Tou Ng Department of Computer Science National University of Singapore 3 Science Drive 2 Singapore 117543 nght@comp.nus.edu.sg Yoong Keok Lee DSO National Laboratories 20 Science Park Drive Singapore 118230 lyoongke@dso.org.sg Abstract In this paper, we present a learning ap- proach to the scenario template task of information extraction, where information filling one template could come from mul- tiple sentences. When tested on the MUC- 4 task, our learning approach achieves accuracy competitive to the best of the MUC-4 systems, which were all built with manually engineered rules. Our analy- sis reveals that our use of full parsing and state-of-the-art learning algorithms have contributed to the good performance. To our knowledge, this is the first re- search to have demonstrated that a learn- ing approach to the full-scale informa- tion extraction task could achieve per- formance rivaling that of the knowledge- engineering approach. 1 Introduction The explosive growth of online texts written in natu- ral language has prompted much research into infor- mation extraction (IE), the task of automatically ex- tracting specific information items of interest from natural language texts. The extracted information is used to fill database records, also known as tem- plates in the IE literature. Research efforts on IE tackle a variety of tasks. They include extracting information from semi- structured texts, such as seminar announcements, rental and job advertisements, etc., as well as from free texts, such as newspaper articles (Soderland, 1999). IE from semi-structured texts is easier than from free texts, since the layout and format of a semi-structured text provide additional useful clues AYACUCHO, 19 JAN 89 – TODAY TWO PEOPLE WERE WOUNDED WHEN A BOMB EXPLODED IN SAN JUAN BAUTISTA MUNICIPALITY. OFFICIALS SAID THAT SHINING PATH MEMBERS WERE RESPONSIBLE FOR THE ATTACK ... ... POLICE SOURCES STATED THAT THE BOMB ATTACK INVOLVING THE SHINING PATH CAUSED SERIOUS DAMAGES ... ... Figure 1: Snippet of a MUC-4 document to aid in extraction. Several benchmark data sets have been used to evaluate IE approaches on semi- structured texts (Soderland, 1999; Ciravegna, 2001; Chieu and Ng, 2002a). For the task of extracting information from free texts, a series of Message Understanding Confer- ences (MUC) provided benchmark data sets for eval- uation. Several subtasks for IE from free texts have been identified. The named entity (NE) task extracts person names, organization names, location names, etc. The template element (TE) task extracts infor- mation centered around an entity, like the acronym, category, and location of a company. The template relation (TR) task extracts relations between enti- ties. Finally, the full-scale IE task, the scenario tem- plate (ST) task, deals with extracting generic infor- mation items from free texts. To tackle the full ST task, an IE system needs to merge information from multiple sentences in general, since the information needed to fill one template can come from multiple sentences, and thus discourse processing is needed. The …

Content truncated to 500 words