The University of Amsterdam at Senseval-3: Semantic roles and Logic forms

The University of Amsterdam at Senseval-3: Semantic Roles and Logic Forms David Ahn Sisay Fissaha Valentin Jijkoun Maarten de Rijke Informatics Institute, University of Amsterdam Kruislaan 403 1098 SJ Amsterdam The Netherlands {ahn,sfissaha,jijkoun,mdr}@science.uva.nl Abstract We describe our participation in two of the tasks or- ganized within Senseval-3: Automatic Labeling of Semantic Roles and Identification of Logic Forms in English. 1 Introduction This year (2004), Senseval, a well-established fo- rum for the evaluation and comparison of word sense disambiguation (WSD) systems, introduced two tasks aimed at building semantic representa- tions of natural language sentences. One task, Auto- matic Labeling of Semantic Roles (SR), takes as its theoretical foundation Frame Semantics (Fillmore, 1977) and uses FrameNet (Johnson et al., 2003) as a data resource for evaluation and system develop- ment. The definition of the task is simple: given a natural language sentence and a target word in the sentence, find other fragments (continuous word sequences) of the sentence that correspond to ele- ments of the semantic frame, that is, that serve as arguments of the predicate introduced by the target word. For this task, the systems receive a sentence, a target word, and a semantic frame (one target word may belong to multiple frames; hence, for real- world applications, a preliminary WSD step might be needed to select an appropriate frame). The out- put of a system is a list of frame elements, with their names and character positions in the sentence. The evaluation of the SR task is based on precision and recall. For this year’s task, the organizers chose 40 frames from FrameNet 1.1, with 32,560 annnotated sentences, 8,002 of which formed the test set. The second task, Identification of Logic Forms in English (LF), is based on the LF formalism de- scribed in (Rus, 2002). The LF formalism is a sim- ple logical form language for natural language se- mantics with only predicates and variables; there is no quantification or negation, and atomic predi- cations are implicitly conjoined. Predicates corre- spond directly to words and are composed of the base form of the word, the part of speech tag, and a sense number (corresponding to the WordNet sense of the word as used). For the task, the system is given sentences and must produce LFs. Word sense disambiguation is not part of the task, so the pred- icates need not specify WordNet senses. System evaluation is based on precision and recall of pred- icates and predicates together with all their argu- ments as compared to a gold standard. 2 Syntactic Processing For both tasks, SR and LF, the core of our systems was the syntactic analysis module described in de- tail in (Jijkoun and de Rijke, 2004). We only have space here to give a short overview of the module. Every sentence was part-of-speech tagged using a maximum entropy tagger (Ratnaparkhi, 1996) and parsed using a state-of-the-art wide coverage phrase structure parser (Collins, 1999). Both the tagger and the parser are trained on the Penn Treebank Wall Street Journal …

Content truncated to 500 words