MathLingBudapest: Concept Networks for Semantic Similarity

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 138–142, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics MathLingBudapest: Concept Networks for Semantic Similarity G´abor Recski Research Institute for Linguistics Hungarian Academy of Sciences Bencz´ur u. 33 1068 Budapest, Hungary recski@mokk.bme.hu Judit ´Acs HAS Research Institute for Linguistics and Dept. of Automation and Applied Informatics Budapest U of Technology and Economics Magyar tud´osok krt. 2 (Bldg. Q.) 1117 Budapest, Hungary judit@mokk.bme.hu Abstract We present our approach to measuring seman- tic similarity of sentence pairs used in Se- meval 2015 tasks 1 and 2. We adopt the sentence alignment framework of (Han et al., 2013) and experiment with several measures of word similarity. We hybridize the common vector-based models with definition graphs from the 4lang concept dictionary and de- vise a measure of graph similarity that yields good results on training data. We did not ad- dress the specific challenges posed by Twitter data, and this is reflected in placing 11th from 30 in Task 1, but our systems perform fairly well on the generic datasets of Task 2, with the hybrid approach placing 11th among 78 runs. 1 Introduction This paper describes the systems participating in Semeval-2015 Task 1 (Xu et al., 2015) and Task 2 (Agirre et al., 2015). To compute the semantic sim- ilarity of two sentences we use the architecture pre- sented in (Han et al., 2013) to find, for each word, its counterpart in the other sentence that is semantically most similar to it. We implemented several meth- ods for measuring word similarity, among them (i) a word embedding created by the method presented in (Mikolov et al., 2013) and (ii) a metric based on networks of concepts derived from the 4lang con- cept lexicon (Kornai and Makrai, 2013; Kornai et al., 2015) and definitions from the Longman Dic- tionary of Contemporary English (Bullon, 2003). A hybrid system exploiting both of these metrics yields the best results and placed 11th among 73 systems on Semeval Task 2a (Semantic Textual Similarity for English). All components of our system are available for download under an MIT license from GitHub12. Section 2 describes the system architec- ture and points out the main differences between our system and that in (Han et al., 2013), section 3 out- lines our word similarity metric derived from the 4lang concept lexicon. We present the evaluation of our systems on both tasks in section 4, and section 5 provides a brief conclusion. 2 Architecture Our framework for determining the semantic simi- larity of two sentences is based on the system pre- sented in (Han et al., 2013). Their architecture, Align and Penalize, involves computing an align- ment score between two sentences based on some measure of word similarity. We’ve chosen to reim- plement this system so that we can experiment with various notions of word similarity, including the one based on 4lang and presented in section 3. Although we reimplemented virtually all rules and components described by (Han et …

Content truncated to 500 words