CMU-AT: Semantic Distance and Background Knowledge for Identifying Semantic Relations

Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages 121–124, Prague, June 2007. c⃝2007 Association for Computational Linguistics CMU-AT: Semantic Distance and Background Knowledge for Identify- ing Semantic Relations Alicia Tribble Language Technologies Institute Carnegie Mellon University Pittsburgh, PA, USA atribble@cs.cmu.edu Scott E. Fahlman Language Technologies Institute Carnegie Mellon University Pittsburgh, PA, USA sef@cs.cmu.edu Abstract This system uses a background knowledge base to identify semantic relations between base noun phrases in English text, as eva- luated in SemEval 2007, Task 4. Training data for each relation is converted to state- ments in the Scone Knowledge Representa- tion Language. At testing time a new Scone statement is created for the sentence under scrutiny, and presence or absence of a relation is calculated by comparing the total semantic distance between the new statement and all positive examples to the total distance between the new statement and all negative examples. 1 Introduction This paper introduces a knowledge-based approach to the task of semantic relation classification, as evaluated in SemEval 2007, Task 4: “Classifying Relations Between Nominals”. In Task 4, a full sentence is presented to the system, along with the WordNet sense keys for two noun phrases which appear there and the name of a semantic relation (e.g. “cause-effect”). The system should return “true” if a person reading the sentence would con- clude that the relation holds between the two la- beled noun phrases. Our system represents a test sentence with a se- mantic graph, including the relation being tested and both of its proposed arguments. Semantic dis- tance is calculated between this graph and a set of graphs representing the training examples relevant to the test sentence. A near-match between a test sentence and a positive training example is evi- dence that the same relation which holds in the example also holds in the test. We compute se- mantic distances to negative training examples as well, comparing the total positive and negative scores in order to decide whether a relation is true or false in the test sentence. 2 Motivation Many systems which perform well on related tasks use syntactic features of the input sentence, coupled with classification by machine learning. This approach has been applied to problems like compound noun interpretation (Rosario and Hearst 2001) and semantic role labeling (Gildea and Ju- rafsky 2002). In preparing our system for Task 4, we started by applying a similar syntax-based feature analysis to the trial data: 140 labeled examples of the rela- tion “content-container”. In 10-fold cross- validation with this data we achieved an average f- score of 70.6, based on features similar to the sub- set trees used for semantic role labeling in (Mo- schitti 2004). For classification we applied the up- dated tree-kernel package (Moschitti 2006), distri- buted with the svm-light tool (Joachims 1999) for learning Support Vector Machines (SVMs). Training data for Task 4 is small, compared to other tasks where machine learning is commonly applied. We had difficulty finding a combination of features which gave good performance …

Content truncated to 500 words