VRep at SemEval-2016 Task 1 and Task 2: A System for Interpretable Semantic Similarity

Proceedings of SemEval-2016, pages 577–583, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics VRep at SemEval-2016 Task 1 and Task 2: A System for Interpretable Semantic Similarity Sam Henry and Allison Sands Computer Science Department Virginia Commonwealth University Richmond, VA 23284, USA henryst@vcu.edu allisonsands1224@gmail.com Abstract VRep is a system designed for SemEval 2016 Task 1 - Semantic Textual Similarity (STS) and Task 2 - Interpretable Semantic Textual Similarity (iSTS). STS quantifies the seman- tic equivalence between two snippets of text, and iSTS provides a reason why those snip- pets of text are similar. VRep makes ex- tensive use of WordNet for both STS, where the Vector relatedness measure is used, and for iSTS, where features are extracted to cre- ate a learned rule-based classifier. This pa- per outlines the VRep algorithm, provides re- sults from the 2016 SemEval competition, and analyzes the performance contributions of the system components. 1 Introduction VRep competed in SemEval 2016 Task 1 - Seman- tic Textual Similarity (STS) and Task 2 - Seman- tic Interpretable Textual Similarity (iSTS). Both of these tasks compute STS between two fragments of text. Task 2 expands upon Task 1 by requiring a rea- son for their similarity. VRep uses an STS measure based on the Vector relatedness measure (Pedersen et al., 2004), and a reasoning system based on JRIP (Cohen, 1995), an implementation of the iREP algo- rithm. For Task 1, we are provided with paired sen- tences, and for each pair of sentences VRep assigns a number indicating their STS. The number ranges from 0 to 5, 0 indicating no similarity and 5 indicat- ing equivalence. For Task 2, we are provided with paired sentences and align the chunks of one sentence to the most similar chunks in the other sentence. Next, a reason and score are computed for that alignment. A chunk is a fragment of text that conveys a single meaning such as in the following example for which chunks are bracketed. [ Black Cow ] [ walking ] [ in a pasture ] [ Black and white Cow ] [ sitting ] [ in the grass ] Alignment reasons are selected from a small list of possible labels created by the event organizers (Agirre et al., 2015): 1. Equivalent (EQUI) - the two chunks convey an equivalent meaning (“hot water”, “scalding wa- ter”) 2. Opposite (OPPO) - the two chunks convey an opposite meaning (“hot water”, “cold water”) 3. More General (SPE1) - this chunk conveys a more general meaning than the other chunk (“hot water”, “water”) 4. More Specific (SPE2) - this chunk conveys a more specific meaning than the other chunk “water”, “hot water”) 5. Similar (SIMI) - the two chunks convey a sim- ilar meaning (“sip water”, “gulp water”) 6. Related (REL) - the two chunks are somehow related (“boil water”, “ocean water”) 7. No Alignment (NOALI) - there are no chunks in the other sentence that are semantically sim- ilar to this chunk 577 As in Task 1 …

Content truncated to 500 words