L2F/INESC-ID at SemEval-2017 Tasks 1 and 2: Lexical and semantic features in word and textual similarity

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 213–219, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics L2F/INESC-ID at SemEval-2017 Tasks 1 and 2: Lexical and semantic features in word and textual similarity Pedro Fialho, Hugo Rodrigues, Lu´ısa Coheur and Paulo Quaresma Spoken Language Systems Lab (L2F), INESC-ID Rua Alves Redol 9 1000-029 Lisbon, Portugal name.surname@l2f.inesc-id.pt Abstract This paper describes our approach to the SemEval-2017 “Semantic Textual Similar- ity” and “Multilingual Word Similarity” tasks. In the former, we test our approach in both English and Spanish, and use a linguistically-rich set of features. These move from lexical to semantic features. In particular, we try to take advantage of the recent Abstract Meaning Representation and SMATCH measure. Although with- out state of the art results, we introduce se- mantic structures in textual similarity and analyze their impact. Regarding word sim- ilarity, we target the English language and combine WordNet information with Word Embeddings. Without matching the best systems, our approach proved to be simple and effective. 1 Introduction In this paper we present two systems that com- peted in SemEval-2017 tasks “Semantic Textual Similarity” and “Multilingual Word Similarity”, using supervised and unsupervised techniques, re- spectively. For the first task we used lexical features, as well as a semantic feature, based in the Ab- stract Meaning Representation (AMR) and in the SMATCH measure. AMR is a semantic formal- ism, structured as a graph (Banarescu et al., 2013). SMATCH is a metric for comparison of AMRs (Cai and Knight, 2013). To the best of our knowl- edge, these were not yet applied to Semantic Tex- tual Similarity. In this paper we focus on the con- tribution of the SMATCH score as a semantic fea- ture for Semantic Textual Similarity, relative to a model based on lexical clues only. For word similarity, we test semantic equiva- lence functions based on WordNet (Miller, 1995) and Word Embeddings (Mikolov et al., 2013). Ex- periments are performed on test data provided in the SemEval-2017 tasks, and yielded compet- itive results, although outperformed by other ap- proaches in the official ranking. The document is organized as follows: in Sec- tion 2 we briefly discuss some related work; in Sections 3 and 4, we describe our systems regard- ing the “Semantic Textual Similarity” and “Mul- tilingual Word Similarity” tasks, respectively. In Section 5 we present the main conclusions and point to future work. 2 Related work The general architecture of our STS system is similar to that of Brychc´ın and Svoboda (2016), Potash et al. (2016) or Tian and Lan (2016), but we employ more lexical features and AMR semantics. Brychc´ın and Svoboda (2016) model feature de- pendence in Support Vector Machines by using the product between pairs of features as new features, while we rely on neural networks. In Potash et al. (2016) it is concluded that feature based systems have better performance than structural learning with syntax trees. A fully-connected neural net- work is employed on hand engineered features and …

Content truncated to 500 words