ConceptNet at SemEval-2017 Task 2: Extending Word Embeddings with Multilingual Relational Knowledge

ConceptNet at SemEval-2017 Task 2: Extending Word Embeddings with Multilingual Relational Knowledge Robyn Speer Luminoso Technologies, Inc. 675 Massachusetts Avenue Cambridge, MA 02139 rspeer@luminoso.com Joanna Lowry-Duda Luminoso Technologies, Inc. 675 Massachusetts Avenue Cambridge, MA 02139 jlowry-duda@luminoso.com Abstract This paper describes Luminoso’s partici- pation in SemEval 2017 Task 2, “Multi- lingual and Cross-lingual Semantic Word Similarity”, with a system based on Con- ceptNet. ConceptNet is an open, multilin- gual knowledge graph that focuses on gen- eral knowledge that relates the meanings of words and phrases. Our submission to SemEval was an update of previous work that builds high-quality, multilingual word embeddings from a combination of Con- ceptNet and distributional semantics. Our system took first place in both subtasks. It ranked first in 4 out of 5 of the separate languages, and also ranked first in all 10 of the cross-lingual language pairs. 1 Introduction ConceptNet 5 (Speer and Havasi, 2013) is a mul- tilingual, domain-general knowledge graph that connects words and phrases of natural language (terms) with labeled, weighted edges. Compared to other knowledge graphs, it avoids trying to be a large gazetteer of named entities. It aims most of all to cover the frequently-used words and phrases of every language, and to represent generally-known relationships between the mean- ings of these terms. The paper describing ConceptNet 5.5 (Speer et al., 2017) showed that it could be used in com- bination with sources of distributional semantics, particularly the word2vec Google News skip-gram embeddings (Mikolov et al., 2013) and GloVe 1.2 (Pennington et al., 2014), to produce new em- beddings with state-of-the-art performance across many word-relatedness evaluations. The three data sources are combined using an extension of the technique known as retrofitting (Faruqui et al., 2015). The result is a system of pre-computed word embeddings we call “ConceptNet Number- batch”. The system we submitted to SemEval-2017 Task 2, “Multilingual and Cross-lingual Semantic Word Similarity”, is an update of that system, co- inciding with the release of version 5.5.3 of Con- ceptNet1. We added multiple fallback methods for assigning vectors to out-of-vocabulary words. We also experimented with, but did not submit, sys- tems that used additional sources of word embed- dings in the five languages of this SemEval task. This task (Camacho-Collados et al., 2017) eval- uated systems at their ability to rank pairs of words by their semantic similarity or relatedness. The words are in five languages: English, German, Italian, Spanish, and Farsi. Subtask 1 compares pairs of words within each of the five languages; subtask 2 compares pairs of words that are in dif- ferent languages, for each of the ten pairs of dis- tinct languages. Our system took first place in both subtasks. Detailed results for our system appear in Sec- tion 3.4. 2 Implementation The way we built our embeddings is based on retrofitting (Faruqui et al., 2015), and in par- ticular, the elaboration of it we call “expanded retrofitting” (Speer et al., 2017). Retrofitting, as originally described, adjusts the values of exist- ing word embeddings based on a …

Content truncated to 500 words