Improving Distantly-supervised Entity Typing with Compact Latent Space Clustering

Proceedings of NAACL-HLT 2019, pages 2862–2872 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2862 Improving Distantly-supervised Entity Typing with Compact Latent Space Clustering Bo Chen1, Xiaotao Gu2, Yufeng Hu1, Siliang Tang1∗, Guoping Hu3, Yueting Zhuang1 & Xiang Ren4 1Zhejiang University, 2University of Illinois at Urbana Champaign 3iFLYTEK Research, 4University of Southern California, {chenbo123, xiaofeem, siliang, yzhuang}@zju.edu.cn, xiaotao2@illinois.edu, gphu@iflytek.com, xiangren@usc.edu Abstract Recently, distant supervision has gained great success on Fine-grained Entity Typing (FET). Despite its efficiency in reducing manual la- beling efforts, it also brings the challenge of dealing with false entity type labels, as dis- tant supervision assigns labels in a context- agnostic manner. Existing works alleviated this issue with partial-label loss, but usually suffer from confirmation bias, which means the classifier fit a pseudo data distribution given by itself. In this work, we propose to regularize distantly supervised models with Compact Latent Space Clustering (CLSC) to bypass this problem and effectively utilize noisy data yet. Our proposed method first dynamically constructs a similarity graph of different entity mentions; infer the labels of noisy instances via label propagation. Based on the inferred labels, mention embeddings are updated accordingly to encourage entity mentions with close semantics to form a com- pact cluster in the embedding space, thus lead- ing to better classification performance. Ex- tensive experiments on standard benchmarks show that our CLSC model consistently out- performs state-of-the-art distantly supervised entity typing systems by a significant margin. 1 Introduction Recent years have seen a surge of interests in fine-grained entity typing (FET) as it serves as an important cornerstone of several nature lan- guage processing tasks including relation extrac- tion (Mintz et al., 2009), entity linking (Raiman and Raiman, 2018), and knowledge base comple- tion (Dong et al., 2014). To reduce manual ef- forts in labelling training data, distant supervision (Mintz et al., 2009) has been widely adopted by recent FET systems. With the help of an external knowledge base (KB), an entity mention is first ∗Corresponding Author. Figure 1: T-SNE visualization of the mention embed- dings generated by NFETC (left) and CLSC (right) on the BBN dataset. Our model (CLSC) clearly groups mentions of the same type into a compact cluster. linked to an existing entity in KB, and then la- beled with all possible types of the KB entity as supervision. However, despite its efficiency, dis- tant supervision also brings the challenge of out- of-context noise, as it assigns labels in a context agnostic manner. Early works usually ignore such noise in supervision (Ling and Weld, 2012; Shi- maoka et al., 2016), which dampens the perfor- mance of distantly supervised models. Towards overcoming out-of-context noise, two lines of work have been proposed to distantly su- pervised FET. The first kind of work try to fil- ter out noisy labels using heuristic rules (Gillick et al., 2014). However, such heuristic pruning sig- nificantly reduces the amount of training data, and thus cannot make full use of distantly annotated data. In contrast, the other thread …

Content truncated to 500 words