Identifying and Explaining Discriminative Attributes

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4313–4322, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4313 Identifying and Explaining Discriminative Attributes Armins Stepanjans and Andr´e Freitas Department of Computer Science University of Manchester armins.stepanjans@student.manchester.ac.uk andre.freitas@manchester.ac.uk Abstract Identifying what is at the center of the mean- ing of a word and what discriminates it from other words is a fundamental natural lan- guage inference task. This paper describes an explicit word vector representation model (WVM) to support the identification of dis- criminative attributes. A core contribution of the paper is a quantitative and qualitative comparative analysis of different types of data sources and Knowledge Bases in the construc- tion of explainable and explicit WVMs: (i) knowledge graphs built from dictionary defini- tions, (ii) entity-attribute-relationships graphs derived from images and (iii) commonsense knowledge graphs. Using a detailed quantita- tive and qualitative analysis, we demonstrate that these data sources have complementary semantic aspects, supporting the creation of explicit semantic vector spaces. The explicit vector spaces are evaluated using the task of discriminative attribute identification, show- ing comparable performance to the state-of- the-art systems in the task (F1-score = 0.69), while delivering full model transparency and explainability. 1 Introduction Word-vector/embedding models (WVM) have emerged as first-class representations in contem- porary Natural Language Processing (NLP) tasks due to their ability to capture semantic similar- ity and relatedness in an unsupervised and com- prehensive manner (Turney and Pantel, 2010), (Freitas, 2015). Additionally, the simplicity en- tailed by the vector space abstraction makes it an engineering-friendly representation, also ex- plaining its widespread adoption and use (Freitas, 2015). However, the latent features (dense vec- tors) at the center of most of the best-performing models have limited their application to two main uses: (i) computing semantic similarity and relat- edness measures and (ii) performing vocabulary generalization as an input layer on Machine Learn- ing (ML) models. The identification of discriminative attributes (IDA), recently introduced by Krebs et al. (2018), can motivate the development of word vector mod- els which can support types of operations with finer semantics, going beyond the computation of semantic similarity and relatedness scores, with potential applications in fine-grained semantic in- ference tasks. Concretely, using the example provided by (Krebs et al., 2018), given a pair of target terms apple and banana, the IDA task seeks to answer if the term red is a discriminative attribute for apple in comparison to banana. According to Krebs et al. semantic difference is a ternary relation between two concepts (apple, banana) and a discriminative feature (red) that characterizes the former concept but not the latter. This paper focuses on proposing an explicit (sparse) WVM for detecting and explaining dis- criminative attributes. The proposed explicit WVM provides a dimension of explainability while keeping the simplicity of the vector space representation model. This addition allows the model to provide a justification while addressing tasks such as the computation of discriminative attributes …

Content truncated to 500 words