Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1790–1799 August 1–6, 2021. ©2021 Association for Computational Linguistics 1790 Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model Hongliang Dai, Yangqiu Song Department of CSE, HKUST {hdai,yqsong}@cse.ust.hk Haixun Wang Instacart haixun.wang@instacart.com Abstract Recently, there is an effort to extend fine- grained entity typing by using a richer and ultra-fine set of types, and labeling noun phrases including pronouns and nomi- nal nouns instead of just named entity men- tions. A key challenge for this ultra-fine en- tity typing task is that human annotated data are extremely scarce, and the annotation abil- ity of existing distant or weak supervision approaches is very limited. To remedy this problem, in this paper, we propose to obtain training data for ultra-fine entity typing by us- ing a BERT Masked Language Model (MLM). Given a mention in a sentence, our approach constructs an input for the BERT MLM so that it predicts context dependent hypernyms of the mention, which can be used as type labels. Ex- perimental results demonstrate that, with the help of these automatically generated labels, the performance of an ultra-fine entity typing model can be improved substantially. We also show that our approach can be applied to im- prove traditional fine-grained entity typing af- ter performing simple type mapping. 1 Introduction Fine-grained entity typing (Ling and Weld, 2012) has been long studied in the natural language pro- cessing community as the extracted type informa- tion is useful for downstream tasks such as entity linking (Ling et al., 2015; Onoe and Durrett, 2020), relation extraction (Koch et al., 2014), coreference resolution (Onoe and Durrett, 2020), etc. Recently, ultra-fine entity typing (Choi et al., 2018) extends the effort to using a richer set of types (e.g., per- son, actor, company, victim) to label noun phrases including not only named entity mentions, but also pronouns and nominal nouns. This task directly uses type words or phrases as tags. Its tag set can contain more than 10,000 types. A challenge is that with the large type set, it is extremely difficult and time-consuming for humans to annotate samples. As a result, most existing works use weak labels that are automatically generated (Ling and Weld, 2012; Choi et al., 2018; Lee et al., 2020). There are two main approaches to obtaining weakly labeled training examples. One approach is to find the Wikipedia pages that correspond to entity mentions, which can be done by using hyper- links to Wikipedia or applying entity linking. Then the entity types can be obtained from knowledge bases. The other approach is to directly use the head words of nominal mentions as ultra-fine type labels. For example, if a nominal mention is “a famous actor,” then the head word “actor” can be used as its type label. Several problems exist when using these weak labels for the ultra-fine typing task. First, in the dataset created by Choi …

Content truncated to 500 words