VENSESEVAL at Semeval-2016 Task 2 iSTS - with a full-fledged rule-based approach

Proceedings of SemEval-2016, pages 796–802, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics VENSESEVAL at Semeval-2016 Task 2: iSTS - with a full-fledged rule- based approach Rodolfo Delmonte Department of Language Studies and Comparative Cultures & Department of Computer Science - Ca’ Foscari University of Venice Ca’ Bembo – Dorsoduro 1075 - 30123 VENEZIA Email:delmont@unive.it - Website: http//project.cgm.unive.it Abstract In our paper we present our rule-based system for semantic processing. In particular we show examples and solutions that may be challenge our approach. We then discuss problems and shortcomings of Task 2 – iSTS. We comment on the existence of a tension between the inherent need to on the one side, to make the task as much as possible “semantically feasible”. Whereas the detailed presentation and some notes in the guidelines refer to inferential processes, paraphrases and the use of commonsense knowledge of the world for the interpretation to work. We then present results and some conclusions. 1 Introduction In this presentation we will focus on Task 2, Interpretable Semantic Textual Similarity. We will comment on the way in which the task has been defined and what is eventually made available to participants to the task. In the section below we will present our system created for RTE – Recognizing Textual Entailment - challenges, and how it has been reorganized to suit the new current task. We discuss examples that suit or challenge our approach. We then comment in detail problems and shortcoming related to issues related to the annotations and the choice of semantic relations. In a final section we will present our results and a discussion. 2 The system VENSESEVAL The system is an adaptation of VENSES Venice team system used for RTE (Recognizing Text Entailment) challenges (see Delmonte et al. 2009). In fact Venses was only partially adaptable to the new task, and so we had to partly recast it. In particular, the semantic evaluation module which was used to issue a binary (or ternary) decision in RTE challenges, in iSTS scenario works in a totally different manner. RTE required a Text - which could be constituted by a single simple or complex sentence - to be compared to an Hypothesis, this usually a simple sentence. From a purely theoretic and abstract point of view, measuring semantic similarity between two sentences (snippets) resembles very closely RTE as indicated in the paper accompanying 2015 challenge, (see Agirre et al. 2015). As the authors comment, iSTS can be defined as a graded similarity notion. In iSTS the comparison is between chunks which must be aligned first. We find this approach certainly very useful and adequate for the task of semantic similarity checking. However, we had to reorganize and partly rewrite the modules for semantic matching. This has taken a lot of time and in fact the module hasn't been fully completed. So we assume that next year challenge will see a better – or at least complete - version of VENSESEVAL. In the current challenge, we …

Content truncated to 500 words