Rev at SemEval-2016 Task 2: Aligning Chunks by Lexical, Part of Speech and Semantic Equivalence

Proceedings of SemEval-2016, pages 777–782, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics Rev at SemEval-2016 Task 2: Aligning Chunks by Lexical, Part of Speech and Semantic Equivalence Tan Ping Ping and Karin Verspoor and Tim Miller Department of Computing and Information System University of Melbourne Australia {pingt@student., karin.verspoor@, tmiller@}unimelb.edu.au Abstract We present the description of our submission to SemEval-2016 Task 2, for the sub-task of aligning pre-annotated chunks between sen- tence pairs and providing similarity and relat- edness labels for the alignment. The objective of the task is to provide interpretable seman- tic textual similarity assessments by adding an explanatory layer to aligned chunks. We anal- ysed the provided datasets, considering lexical overlap, the part of speech tags and the syn- onyms of the words in the chunks, and devel- oped a rule-based system reflecting that anal- ysis. Our system performance indicates that when sentence pairs are similar, alignment of chunks can be performed fairly well using lex- ical information alone without syntactic or se- mantic analysis. The advantage of our system is that we can easily trace when chunks are aligned. 1 Introduction We developed a system for SemEval-2016 Task 2: “Interpretable Semantic Textual Similarity” (Agirre et al., 2016), which requires the development of a system that labels aligned chunks of a sentence pair, in terms of their similarity or relatedness. Our ap- proach is based on a detailed analysis of the pro- vided annotation guidelines for the task and the an- notated training data. The annotation guidelines for the task indicate how to align the chunks of two sentence pairs and how to label the similarity or relatedness types and assign scores. Annotations are provided for three different training sets: headlines, images and student responses to questions. Chunks consist of a word or contiguous words. The training sets come in the form of sentence pairs, pre-annotated chunks with their alignment and alignment labels. The number of sentence pairs for the training and test sets supplied is, respectively: headlines – 726 and 375; images – 750 and 375; and student an- swers – 330 and 344. The test sets come with the pre-annotated chunks but without alignments or la- bels. Participants of the sub-task are required to align the pre-annotated chunks and provide the sim- ilarity and relatedness assessment for the alignment, which is considered as an explanatory layer that en- riches measurement of similarity between the sen- tence pairs. The alignment types are semantically equivalent (EQUI), opposite in meaning (OPPO), similar in meaning but specific to the chunk in the first sen- tence (SPE1), similar in meaning but specific to the chunk in second sentence (SPE2), similar in meaning (SIMI) and related in meaning (REL). Ad- ditional types, are factuality (FACT) and polarity (POL). For chunks with no alignment, the label is NOALI. The similarity and relatedness scores are 5 for equivalent (i.e. EQUI//5), [4, 3] for very sim- ilar or closely related, [2, 1] for slightly similar or somehow related and 0 for …

Content truncated to 500 words