VecShare: A Framework for Sharing Word Representation Vectors

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 316–320 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics VecShare: A Framework for Sharing Word Representation Vectors Jared Fernandez and Zhaocheng Yu and Doug Downey Department of Electrical Engineering and Computer Science Northwestern University Evanston, IL 60208 {jared.fern|zhaochengyu2017}@u.northwestern.edu ddowney@eecs.northwestern.edu Abstract Many Natural Language Processing (NLP) models rely on distributed vector representations of words. Because the process of training word vectors can require large amounts of data and compu- tation, NLP researchers and practitioners often utilize pre-trained downloaded from the Web.embeddings However, finding the best embeddings for a given task is difficult, and can be computation- ally prohibitive. We present a framework, called VecShare, that makes it easy to share and retrieve word embeddings on the Web. The framework leverages a public data-sharing infrastructure to host embedding sets, and provides automated mechanisms for retrieving the embed- dings most similar to a given corpus. We perform an experimental evaluation of VecShare’s similarity strategies, and show that they are effective at efficiently retrieving embeddings that boost accuracy in a document classification task. Finally, we provide an open-source Python library for using the VecShare framework.1 1 Introduction Word embeddings capture syntactic and semantic properties of words, and are a key component of many modern NLP models (Turian et al., 2010). However, high-quality embeddings can be expen- sive to train. As a result, rather than training their own embeddings, NLP researchers and practition- ers often download pre-trained embeddings from the Web, e.g. (Limsopatham and Collier, 2016; Cheng et al., 2016). 1https://github.com/JaredFern/VecShare However, existing methods for sharing embed- dings on the Web are suboptimal. Current prac- tice primarily consists of contributors posting em- bedding sets to their own Web sites. No central embedding repository exists, and it is difficult for users to know which embedding sets are available. Furthermore, determining the utility of an embed- ding set for a particular NLP task entails signifi- cant time and computational costs, as users must manually download and evaluate multiple com- plete embedding sets. Methods exist for automati- cally scoring an embedding set, but these are lim- ited to specific tasks and lack integration with ex- isting code bases (Faruqui and Dyer, 2014). Our goal in this paper is to introduce a frame- work that makes sharing word embeddings easier for NLP researchers and practitioners. It should be simple and fast to post, browse, and retrieve em- beddings from a public data store. Additionally, integration with existing NLP codebases should be more seamless: software libraries should automat- ically identify and download the particular embed- dings that are likely to be relevant to a user’s cor- pus. This paper presents VecShare, a framework for sharing word embeddings. As its data store, it uses an existing public data-sharing platform, which provides searching and browsing capabil- ity. To solve the critical challenge of helping users quickly find relevant embeddings, we in- troduce embedding indexers. The indexers com- pute and share compact representations, called sig- natures, for each …

Content truncated to 500 words