Evaluating Various Linguistic Features on Semantic Relation Extraction

Proceedings of Recent Advances in Natural Language Processing, pages 721–726, Hissar, Bulgaria, 12-14 September 2011. Evaluating Various Linguistic Features on Semantic Relation Extraction Marcos Garcia Center for Research in Information Technologies (CITIUS) University of Santiago de Compostela marcos.garcia.gonzalez@usc.es Pablo Gamallo Center for Research in Information Technologies (CITIUS) University of Santiago de Compostela pablo.gamallo@usc.es Abstract Machine learning approaches for Information Extraction use different types of features to acquire semantically related terms from free text. These features may contain several kinds of linguistic knowledge: from orthographic or lexical to more complex features, like PoS- tags or syntactic dependencies. In this paper we select four main types of linguistic features and evaluate their performance in a system- atic way. Despite the combination of some types of features allows us to improve the f- score of the extraction, we observed that by adjusting the positive and negative ratio of the training examples, we can build high quality classifiers with just a single type of linguis- tic feature, based on generic lexico-syntactic patterns. Experiments were performed on the Portuguese version of Wikipedia. 1 Introduction With the exponential growth of data, the interest in learning semantic information related to named en- tities has been increased. For instance, from the sen- tence Ernest Hemingway (July 21, 1899 - July 2, 1961) was an American author, a system may learn various properties about Hemingway (his birth and death dates, his origin as well as his occupation). Many techniques employ Machine Learning (ML) algorithms for the extraction task. They ar- range into a set of features the contexts or sentences in which pairs of related entities occur. These fea- tures are then used to train a classifier. The start- ing point is a number of labeled training examples (“Ernest Hemingway - author”), as well as a corpus- based strategy to identify and represent as features those sentences (contexts) in which the training ex- amples occur. These techniques may have different degrees of supervision, from manually constructed corpora to weakly supervised on unsupervised meth- ods. Moreover, it must be pointed out that features can be represented at different levels of generality, according to different types of knowledge. However, there are no much work on the importance of know- ing what linguistic information is actually useful in order to increase the performance of these systems. In this article, we evaluate and compare the im- pact of different types of linguistic features for Re- lation Extraction (RE). We built and tested a distant supervision system with different types of linguis- tic features: bags of lemmas and PoS-tags, lexico- syntactic patterns and syntactic dependencies. We evaluated the performance of these types of features individually and in several combinations. Prelimi- nary results in Portuguese data show that, usually, the combination of some types of linguistic features allows us to increase recall without losing precision. Furthermore, we also observed that a deep analysis of the positive/negative (P/N) ratio of the training examples improves the f-score of the classifiers. Sect. 2 introduces some related work. Then, …

Content truncated to 500 words