Using Uneven Margins SVM and Perceptron for Information Extraction

Proceedings of the 9th Conference on Computational Natural Language Learning (CoNLL), pages 72–79, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Using Uneven Margins SVM and Perceptron for Information Extraction Yaoyong Li, Kalina Bontcheva and Hamish Cunningham Department of Computer Science, The University of Sheffield, Sheffield, S1 4DP, UK {yaoyong,kalina,hamish}@dcs.shef.ac.uk Abstract The classification problem derived from information extraction (IE) has an imbal- anced training set. This is particularly true when learning from smaller datasets which often have a few positive training examples and many negative ones. This paper takes two popular IE algorithms – SVM and Perceptron – and demonstrates how the introduction of an uneven margins parameter can improve the results on im- balanced training data in IE. Our experi- ments demonstrate that the uneven margin was indeed helpful, especially when learn- ing from few examples. Essentially, the smaller the training set is, the more bene- ficial the uneven margin can be. We also compare our systems to other state-of-the- art algorithms on several benchmarking corpora for IE. 1 Introduction Information Extraction (IE) is the process of auto- matic extraction of information about pre-specified types of events, entities or relations from text such as newswire articles or Web pages. IE is useful in many applications, such as information gathering in a variety of domains, automatic annotations of web pages for Semantic Web, and knowledge manage- ment. A wide range of machine learning techniques have been used for IE and achieved state-of-the-art results, comparable to manually engineered IE sys- tems. A learning algorithm usually learns a model from a set of documents which have been manually annotated by the user. Then the model can be used to extract information from new documents. Manual annotation is a time-consuming process. Hence, in many cases learning from small data sets is highly desirable. Therefore in this paper we also evaluate the performance of our algorithms on small amounts of training data and show their learning curve. The learning algorithms for IE can be classified broadly into two main categories: rule learning and statistical learning. The former induces a set of rules from training examples. There are many rule based learning systems, e.g. SRV (Freitag, 1998), RAPIER (Califf, 1998), WHISK (Soderland, 1999), BWI (Freitag and Kushmerick, 2000), and (LP)2 (Ciravegna, 2001). Statistical systems learn a statis- tical model or classifiers, such as HMMs (Freigtag and McCallum, 1999), Maximal Entropy (Chieu and Ng., 2002), the SVM (Isozaki and Kazawa, 2002; Mayfield et al., 2003), and Perceptron (Carreras et al., 2003). IE systems also differ from each other in the NLP features that they use. These include simple features such as token form and capitalisa- tion information, linguistic features such as part-of- speech, semantic information from gazetteer lists, and genre-specific information such as document structure. In general, the more features the system uses, the better performance it can achieve. This paper concentrates on classifier-based learn- ing for IE, which typically converts the recognition of each information entity into a set of classification problems. In the …

Content truncated to 500 words