Turning Distributional Thesauri into Word Vectors for Synonym Extraction and Expansion

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 273–283, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Turning Distributional Thesauri into Word Vectors for Synonym Extraction and Expansion Olivier Ferret CEA, LIST, Vision and Content Engineering Laboratory, Gif-sur-Yvette, F-91191 France. olivier.ferret@cea.fr Abstract In this article, we propose to investigate a new problem consisting in turning a distributional thesaurus into dense word vectors. We propose more precisely a method for performing such task by asso- ciating graph embedding and distributed representation adaptation. We have ap- plied and evaluated it for English nouns at a large scale about its ability to re- trieve synonyms. In this context, we have also illustrated the interest of the devel- oped method for three different tasks: the improvement of already existing word em- beddings, the fusion of heterogeneous rep- resentations and the expansion of synsets. 1 Introduction Early work about distributional semantics (Grefenstette, 1994; Lin, 1998; Curran and Moens, 2002) was strongly focused on the notion of distributional thesaurus. Recent work in this domain has been more concerned by the notions of semantic similarity and relatedness (Budanitsky and Hirst, 2006) and by the representation of dis- tributional data. This trend has been strengthened even more recently with all work about distributed word representations and embeddings, whether they are built by neural networks (Mikolov et al., 2013) or not (Pennington et al., 2014). From a more global perspective, distributional thesauri and distributional data, i.e. distributional contexts of words, can be considered as dual rep- resentations of the same semantic similarity infor- mation. Distributional data are an intensional form of this information that can take an extensional form as distributional thesauri by applying a simi- larity measure to them. Going from an intensional to an extensional representation corresponds to the distributional context wi word mi word mk distributional context wk entries w1 … wi … wn neighbors thesaurus embedding thesaurus building Figure 1: Duality of semantic information rather classical process underlying the building of distributional thesauri. In the context of word em- beddings, Perozzi et al. (2014a) extend this pro- cess to the building of lexical networks. Going to the other way, from an extensional to an intensional representation, is, as far as we know, a new problem in the context of distribu- tional semantics. The interest of this transforma- tion is twofold. First, whatever the initial form of the semantic knowledge, it can be turned into the most suitable form for a particular use. For instance, thesauri are more suitable for tasks like query expansion while word embeddings are more adapted as features for statistical classifiers. Sec- ond, each form is also associated with specific methods of improvement. A lot of work has been done for improving distributional contexts by studying various parameters, which has led to an important improvement of distributional the- sauri. Conversely, work such as (Claveau et al., 2014) has focused on methods for improving the- sauri themselves. It would clearly be interesting to transpose the improvements …

Content truncated to 500 words