Unsupervised Word Mapping Using Structural Similarities in Monolingual Embeddings

Unsupervised Word Mapping Using Structural Similarities in Monolingual Embeddings Hanan Aldarmaki Mahesh Mohan Mona Diab Department of Computer Science The George Washington University {aldarmaki;mahesh_mohan;mtdiab}@gwu.edu Abstract Most existing methods for automatic bilingual dictionary induction rely on prior alignments between the source and target languages, such as parallel corpora or seed dictionaries. For many language pairs, such supervised align- ments are not readily available. We propose an unsupervised approach for learning a bilingual dictionary for a pair of languages given their independently-learned monolingual word em- beddings. The proposed method exploits lo- cal and global structures in monolingual vector spaces to align them such that similar words are mapped to each other. We show empir- ically that the performance of bilingual cor- respondents that are learned using our pro- posed unsupervised method is comparable to that of using supervised bilingual correspon- dents from a seed dictionary. 1 Introduction The working hypothesis in distributional semantics is that the meaning of a word can be inferred by its distribution, or co-occurrence, around other words. The validity of this hypothesis is most evident in the performance of distributed vector representations of words, i.e word embeddings, that are automati- cally induced from large text corpora (Bengio et al., 2003; Mikolov et al., 2013b). The qualitative nature of these embeddings can be demonstrated through empirical evidence of regularities that reflect certain semantic relationships. Words in the vector space are generally clustered by meaning, and the dis- tances between words and clusters reflect semantic or syntactic relationships, which makes it possible to perform arithmetic on word vectors for analog- ical reasoning and semantic composition (Mikolov et al., 2013b). For example, in a vector space V where f = V (‘‘france”), p = V (‘‘paris”), and g = V (‘‘germany”), the distance f − p reflects the country-capital relationship, and g + f − p results in a vector closest to V (‘‘berlin”). Named entities and inflectional morphemes are particularly amenable to vector arithmetic, while derivational morphology, polysemy, and other nuanced semantic categories re- sult in lower performance in analogy questions (Fin- ley et al., 2017). The extent of these semantic and syntactic regu- larities is difficult to assess intrinsically, and the per- formance in analogical reasoning can be partially at- tributed to the clustering of the words in question (Linzen, 2016). If meaning is encoded in the rela- tive distances among word vectors, then the struc- ture within vector spaces should be consistent across different languages given that the datasets used to build them express similar content. In Rapp (1995), a simulation study showed that similarity in word co-occurrence patterns within unrelated German and English texts is correlated with the number of cor- responding word positions in the monolingual co- occurrence matrices. More recently, Mikolov et al. (2013a) showed that a linear projection can be learned to transform word embeddings from one language into the vector space of another using a medium-size seed dictionary, which demonstrates that the multilingual vector spaces are at least re- lated by a linear …

Content truncated to 500 words