UBC: Cubes for English Semantic Textual Similarity and Supervised Approaches for Interpretable STS

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 178–183, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics UBC: Cubes for English Semantic Textual Similarity and Supervised Approaches for Interpretable STS Eneko Agirre, Aitor Gonzalez-Agirre, I˜nigo Lopez-Gazpio, Montse Maritxalar, German Rigau, Larraitz Uria University of the Basque Country Donostia, 20018, Basque Country {e.agirre, aitor.gonzalez-agirre, inigo.lopez, montse.maritxalar, german.rigau, larraitz.uria}@ehu.eus Abstract In Semantic Textual Similarity, systems rate the degree of semantic equivalence on a graded scale from 0 to 5, with 5 being the most similar. For the English subtask, we present a system which relies on several resources for token-to-token and phrase-to-phrase similarity to build a data-structure which holds all the in- formation, and then combine the information to get a similarity score. We also participated in the pilot on Interpretable STS, where we ap- ply a pipeline which first aligns tokens, then chunks, and finally uses supervised systems to label and score each chunk alignment. 1 Introduction In Semantic Textual Similarity (STS), systems rate the degree of semantic equivalence on a graded scale from 0 to 5, with 5 being the most similar. We partic- ipated in two of the subtask for STS in 2015 (Agirre et al., 2015). For the English subtask, we present a system which relies on several resources for token- to-token and phrase-to-phrase similarity to build a data-structure which holds all the information, and then combine the information to get a similarity score. We also participated in the pilot on Inter- pretable STS, where we apply a pipeline which first aligns tokens, then chunks, and finally uses super- vised systems to label and score each chunk align- ment. Note that some of the authors participated in the organization of the task. We scrupulously separated the tasks in such a way that the developers of the systems did not have access to the test sets, and that they only had access to the same training data as the rest of the participants. 2 Cubes for English STS In this section we describe a novel approach to com- pute similarity scores between two sentences using a cube where each layer contains token-to-token and phrase-to-phrase similarity scores from a different method and/or resource. Our assumption is that we can obtain better results using this similarity scores together than independently. 2.1 Building Cubes The first step is to produce parse trees for the sen- tences using the Stanford Parser (Toutanova et al., 2003). After parsing the sentences each pair of sen- tences can be represented by a NxM matrix, being N is the number of nodes of the parse tree of the first sentence, and M the number of nodes of the parse tree of the second sentence. Note that some nodes (terminals) correspond to words, while others (non- terminals) represent phrases. We can have as many matrices as we wish, and fill them with different sim- ilarity scores, forming a cube. In this first attempt we used three layers: 1. Euclidean distance between Collobert …

Content truncated to 500 words