Social Image Tags as a Source of Word Embeddings: A Task-oriented Evaluation

Social Image Tags as a Source of Word Embeddings: A Task-oriented Evaluation Mika Hasegawa, Tetsunori Kobayashi, Yoshihiko Hayashi School of Science and Engineering, Waseda University Waseda-machi 27, Shinjuku, Tokyo 1690042, Japan mika@pcl.cs.waseda.ac.jp, koba@waseda.jp, yshk.hayashi@aoni.waseda.jp Abstract Distributional hypothesis has been playing a central role in statistical NLP. Recently, however, its limitation in incorporating perceptual and empirical knowledge is noted, eliciting a field of perceptually grounded computational semantics. Typical sources of features in such a research are image datasets, where images are accompanied by linguistic tags and/or descriptions. Mainstream approaches employ ma- chine learning techniques to integrate/combine visual features with linguistic features. In contrast to or supplementing these approaches, this study assesses the effectiveness of social image tags in generating word embeddings, and argues that these generated representa- tions exhibit somewhat different and favorable behaviors from corpus-originated representations. More specifically, we generated word embeddings by using image tags obtained from a large social image dataset YFCC100M, which collects Flickr images and the associ- ated tags. We evaluated the efficacy of generated word embeddings with standard semantic similarity/relatedness tasks, which showed that comparable performances with corpus-originated word embeddings were attained. These results further suggest that the generated embeddings could be effective in discriminating synonyms and antonyms, which has been an issue in distributional hypothesis-based approaches. In summary, social image tags can be utilized as yet another source of visually enforced features, provided the amount of available tags is large enough. Keywords: word embeddings, image tags, social media, semantic similarity, synonyms, antonyms. 1. Introduction Virtually, all the methods for generating distribu- tional/distributed word representations (Baroni et al., 2014) rely on the notion of distributional hypothesis (Firth, 1957). These approaches enable word representations to properly capture the distributional hypothesis by measuring the com- monality of the linguistic contexts of word occurrences. Al- though these approaches are proven effective in various se- mantic tasks, they are limited in terms of the incorporation of perceptual and empirical knowledge: perceptually or empirically obvious objects have not been necessarily well verbalized in a corpus of written texts (Bruni et al., 2014). Yet another issue with the distributional hypothesis-based methods is that they often run into trouble when discrim- inating synonyms from antonyms or more vaguely related words (Hill et al., 2015). Recently, motivated by these issues, several research works that try to incorporate human perceptual/empirical knowledge into linguistically derived representations have emerged. Most typically, such approaches com- bine/integrate visual features achieved from visual re- sources with linguistic features (word embeddings) by ap- plying machine learning/deep learning techniques. To en- able this line of research, a visual resource in which an im- age is accompanied by linguistic descriptions is generally required. Although these methods compensated/improved purely lin- guistic representations, the source of visual features cannot be limited to image data. That is, if a content of any modal- ity is described with a substantial amount of linguistic tags and/or descriptions, the linguistic co-occurrence observed around the content can be utilized as a source of semantic features. Once resting on this …

Content truncated to 500 words