Word Attribute Prediction Enhanced by Lexical Entailment Tasks

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5846–5854 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5846 Word Attribute Prediction Enhanced by Lexical Entailment Tasks Mika Hasegawa, Tetsunori Kobayashi, Yoshihiko Hayashi Faculty of Science and Engineering, Waseda University Waseda-machi 27, Shinjuku, Tokyo 169-0042, Japan mika@pcl.cs.waseda.ac.jp, koba@waseda.jp, yshk.hayashi@aoni.waseda.jp Abstract Human semantic knowledge about concepts acquired through perceptual inputs and daily experiences can be expressed as a bundle of attributes. Unlike the conventional distributed word representations that are purely induced from a text corpus, a semantic attribute is associated with a designated dimension in attribute-based vector representations. Thus, semantic attribute vectors can effectively capture the commonalities and differences among concepts. However, as semantic attributes have been generally created by psychological experimental settings involving human annotators, an automatic method to create or extend such resources is highly demanded in terms of language resource development and maintenance. This study proposes a two-stage neural network architecture, Word2Attr, in which initially acquired attribute representations are then fine-tuned by employing supervised lexical entailment tasks. The quantitative empirical results demonstrated that the fine-tuning was indeed effective in improving the performances of semantic/visual similarity/relatedness evaluation tasks. Although the qualitative analysis confirmed that the proposed method could often discover valid but not-yet human-annotated attributes, they also exposed future issues to be worked: we should refine the inventory of semantic attributes that currently relies on an existing dataset. Keywords: word attributes, word attribute prediction, lexical entailment, similarity, fine-tuning 1. Introduction A semantic attribute of a concept, such as “an apple is red”, explicitly dictates a semantic aspect of the concept. Thus, given an appropriate set of attributes, a concept can be represented by a bundle of attributes. This notion of se- mantic attribute (often referred to as semantic feature or se- mantic property) has some connections with the componen- tial analysis in linguistics. A trait of the semantic attributes that they are associated with human percepts and experi- ences is solidly emphasized by psychological theories, in particular by the notion of grounded cognition (Barsalou, 2008). Unlike the conventional distributed representa- tions (Mikolov et al., 2013; Pennington et al., 2014; Bojanowski et al., 2016) purely induced from a text corpus, a semantic attribute is linked with a designated dimension in the vector representations, making their representation appropriate for the computation of commonalities and differences among concepts (Lazaridou et al., 2016; Krebs et al., 2018). This good property could lead to improve the interpretability of an attribute-based NLP system. Moreover, if the semantic attributes even for an unseen concept can be adequately predicted by employing auxiliary information, these attributes can be effectively utilized in several semantic tasks including zero-shot learning/recognition (Al-Halah et al., 2016; Li et al., 2018). In this regard, visual semantic attributes, proven effective in the area of computer vision and the application systems (Lampert et al., 2014; Feris et al., 2017), could provide useful auxiliary information. However, as semantic attributes are generally created by psychological experimental settings involving human …

Content truncated to 500 words