Imposing Label-Relational Inductive Bias for Extremely Fine-Grained Entity Typing

Proceedings of NAACL-HLT 2019, pages 773–784 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 773 Imposing Label-Relational Inductive Bias for Extremely Fine-Grained Entity Typing Wenhan Xiong†, Jiawei Wu†, Deren Lei†, Mo Yu∗, Shiyu Chang∗, Xiaoxiao Guo∗, William Yang Wang† † University of California, Santa Barbara ∗ IBM Research {xwhan, william}@cs.ucsb.edu, yum@us.ibm.com, {shiyu.chang, xiaoxiao.guo}@ibm.com Abstract Existing entity typing systems usually ex- ploit the type hierarchy provided by knowl- edge base (KB) schema to model label cor- relations and thus improve the overall perfor- mance. Such techniques, however, are not di- rectly applicable to more open and practical scenarios where the type set is not restricted by KB schema and includes a vast number of free-form types. To model the underly- ing label correlations without access to man- ually annotated label structures, we introduce a novel label-relational inductive bias, repre- sented by a graph propagation layer that effec- tively encodes both global label co-occurrence statistics and word-level similarities. On a large dataset with over 10,000 free-form types, the graph-enhanced model equipped with an attention-based matching module is able to achieve a much higher recall score while main- taining a high-level precision. Specifically, it achieves a 15.3% relative F1 improvement and also less inconsistency in the outputs. We fur- ther show that a simple modification of our proposed graph layer can also improve the per- formance on a conventional and widely-tested dataset that only includes KB-schema types.1 1 Introduction Fine-grained entity typing is the task of identifying specific semantic types of entity mentions in given contexts. In contrast to general entity types (e.g., organization, event), fine-grained types (e.g., po- litical party, natural disaster) are often more infor- mative and can provide valuable prior knowledge for a wide range of NLP tasks, such as coreference resolution (Durrett and Klein, 2014), relation ex- traction (Yaghoobzadeh et al., 2016) and question answering (Lee et al., 2006; Yavuz et al., 2016). 1https://github.com/xwhan/ Extremely-Fine-Grained-Entity-Typing Context Types Big Show then appeared at One Night Stand, attacking Tajiri, Super Crazy, and the Full Blooded Italians after their tag team match person†, televi- sion program⋆ person, athlete, wrestler, entertainer The womens pole vault at the 2010 IAAF World Indoor Championships was held at the ASPIRE Dome on 12 and 14 March. month†, event⋆ date, month Table 1: Examples of inconsistent predictions pro- duced by existing entity typing system that does not model label correlations. We use different subscript symbols to indicate contradictory type pairs and show the ground-truth types in italics. In practical scenarios, a key challenge of entity typing is to correctly predict multiple ground-truth type labels from a large candidate set that covers a wide range of types in different granularities. In this sense, it is essential for models to effectively capture the inter-label correlations. For instance, if an entity is identified as a “criminal”, then the en- tity must also be a “person”, but it is less likely for this entity to be a “police officer” at the same time. When ignoring such …

Content truncated to 500 words