An Attentive Fine-Grained Entity Typing Model with Latent Type Representation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 6197–6202, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 6197 An Attentive Fine-Grained Entity Typing Model with Latent Type Representation Ying Lin and Heng Ji Computer Science Department University of Illinois at Urbana-Champaign, Urbana, IL, USA 61802 {yinglin8,hengji}@illinois.edu Abstract We propose a fine-grained entity typing model with a novel attention mechanism and a hy- brid type classifier. We advance existing meth- ods in two aspects: feature extraction and type prediction. To capture richer contextual infor- mation, we adopt contextualized word repre- sentations instead of fixed word embeddings used in previous work. In addition, we propose a two-step mention-aware attention mecha- nism to enable the model to focus on im- portant words in mentions and contexts. We also present a hybrid classification method be- yond binary relevance to exploit type inter- dependency with latent type representation. Instead of independently predicting each type, we predict a low-dimensional vector that en- codes latent type features and reconstruct the type vector from this latent representation. Ex- periment results on multiple data sets show that our model significantly advances the state- of-the-art on fine-grained entity typing, obtain- ing up to 6.6% and 5.5% absolute gains in macro averaged F-score and micro averaged F- score respectively. 1 1 Introduction Fine-grained entity typing aims to assign one or more types to each entity mention given a certain context. For example, in the following sentence, “If Rogers is in the game, the Huskies will be much better equipped to match the Cougars in that as- pect”, the mention “Rogers” should be labeled as athlete in addition to person according to the context (e.g., game, Huskies). These fine-grained entity types are proven to be effective in support- ing a wide range of downstream applications such as relation extraction (Yao et al., 2010), question answering (Lin et al., 2012), and coreference res- olution (Recasens et al., 2013). 1Code for this paper is available at: https://github. com/limteng-rpi/fet. Fine-grained entity typing is usually formulated as a multi-label classification problem. Previous approaches (Ling and Weld, 2012; Choi et al., 2018; Xin et al., 2018) typically address it with bi- nary relevance that decomposes the problem into isolated binary classification subproblems and in- dependently predicts each type. However, this method is commonly criticized for its label inde- pendence assumption, which is not valid for fine- grained entity typing. For example, if the model is confident at predicting the type artist, it should promote its parent type person but discourage organization and its descendant types. In or- der to capture inter-dependencies between types, we propose a hybrid model that incorporates la- tent type representation in addition to binary rel- evance. Specifically, the model learns to predict a low-dimensional vector that encodes latent type features obtained through Principle Label Space Transformation (Tai and Lin, 2012) and recon- struct the sparse and high-dimensional type vector from this latent representation. Another major …

Content truncated to 500 words