Comparing Pretrained Multilingual Word Embeddings on an Ontology Alignment Task

Comparing Pretrained Multilingual Word Embeddings on an Ontology Alignment Task Dagmar Gromann, Thierry Declerck Technical University Dresden, DFKI GmbH N¨othnitzer Str. 46, D-01187 Dresden, Germany, Stuhlsatzenhausweg 3, D-66123, Saarbr¨ucken, Germany, dagmar.gromann@gmail.com, declerck@dfki.de Abstract Word embeddings capture a string’s semantics and go beyond its surface form. In a multilingual environment, those embeddings need to be trained for each language, either separately or as a joint model. The more languages needed, the more computationally cost- and time-intensive the task of training. As an alternative, pretrained word embeddings can be utilized to compute semantic similarities of strings in different languages. This paper provides a comparison of three different multilingual pretrained word embedding repositories with a string-matching baseline and uses the task of ontology alignment as example scenario. A vast majority of ontology alignment methods rely on string similarity metrics, however, they frequently use string matching techniques that purely rely on syntactic aspects. Semantically oriented word embeddings have much to offer to ontology alignment algorithms, such as the simple Munkres algorithm utilized in this paper. The proposed approach produces a number of correct alignments on a non-standard data set based on embeddings from the three repositories, where FastText embeddings performed best on all four languages and clearly outperformed the string-matching baseline. Keywords: word embeddings, ontology alignment, multilingual resources, distributional semantics, comparison, evaluation 1. Introduction Word embeddings constitute a distributed word representa- tion to leverage the semantics of words by mapping them to vectors of real numbers, where each dimension of the embedding represents a latent feature of the word (Turian et al., 2010). They have been shown to be very success- ful in many NLP tasks (Mikolov et al., 2013; Camacho- Collados et al., 2016) and also ontology alignment (Zhang et al., 2014). Evaluations of embeddings mostly focus on English standard datasets with high frequency single words (Baroni et al., 2014; Riedl and Biemann, 2017) and the few available multilingual comparisons, such as by Camacho- Collados et al. (2016), usually focus on one type of em- bedding. This paper proposes the use of a non-standard, domain-specific and multilingual dataset with multi-word expressions to compare three different pretrained embed- ding repositories. Re-use of existing embeddings is attrac- tive since no training time or expertise in learning embed- dings is required. Embeddings abstract away from the string’s surface form, which is not the case with syntactic similarity metrics, such as the Levenshtein edit distance (Levenshtein, 1966), conventionally used in ontology alignment (see Cheatham and Hitzler (2013) for a comparison). A semantic repre- sentation of words is important in ontology alignment for two major reasons. First, it allows to consider synonyms and terminological variants. For instance, “Schuhe”@de (shoes) should have a lower similarity when compared to “Schule”@de (school) than compared to its actual syn- onym “Fußbekleidung”@de (footwear). Second, shortened strings, such as abbreviations, co-occur in the same con- text as their full form but differ strongly in their surface form. However, the computational cost of training word embeddings increases proportionally with a rising number of languages considered. …

Content truncated to 500 words