Global Bootstrapping Neural Network for Entity Set Expansion

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3705–3714 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3705 Global Bootstrapping Neural Network for Entity Set Expansion Lingyong Yan1,3, Xianpei Han1,2,∗, Ben He3,1, Le Sun1,2 1Chinese Information Processing Laboratory 2State Key Laboratory of Computer Science Institute of Software, Chinese Academy of Sciences, Beijing, China 3University of Chinese Academy of Sciences, Beijing, China {lingyong2014, xianpei}@iscas.ac.cn, benhe@ucas.ac.cn, sunle@iscas.ac.cn Abstract Bootstrapping for entity set expansion (ESE) has been studied for a long period, which ex- pands new entities using only a few seed en- tities as supervision. Recent end-to-end boot- strapping approaches have shown their advan- tages in information capturing and bootstrap- ping process modeling. However, due to the sparse supervision problem, previous end- to-end methods often only leverage informa- tion from near neighborhoods (local seman- tics) rather than those propagated from the co-occurrence structure of the whole corpus (global semantics). To address this issue, this paper proposes Global Bootstrapping Network (GBN) with the “pre-training and fine-tuning” strategies for effective learning. Specifically, it contains a global-sighted encoder to capture and encode both local and global semantics into entity embedding, and an attention-guided decoder to sequentially expand new entities based on these embeddings. The experimen- tal results show that the GBN learned by “pre- training and fine-tuning” strategies achieves state-of-the-art performance on two bootstrap- ping datasets. 1 Introduction Bootstrapping is a classical technique for entity set expansion (ESE), which starts from several seed entities of a specific category (e.g., {London, Beijing, U.S.} for GPE category) and then it- eratively expands the entity set to cover more en- tities of the category (e.g., Egypt and Harare). Most previous ESE studies (Riloff and Jones, 1999; Curran et al., 2007; Yan et al., 2019) adopt the pipelined paradigm (see Figure 1a), which itera- tively: evaluates patterns using seeds, matches and evaluates entities using patterns, adds top entities to the seed set. Such a pipelined paradigm makes it hard to represent the whole bootstrapping process *Corresponding author. ... Corpus Encoder Decoder Seed Entities Expansion Step=1 Expansion Step=2 Expansion Step=3 (b). end-to-end (a). pipelined Pattern Evaluation Entity Evaluation Entity Matching Seed Entities evaluated patterns entities & patterns top evaluated entities Figure 1: The illustration of the pipelined (a) and the end-to-end (b) bootstrapping paradigm. as a single learnable model, and the implementa- tion of ESE systems are often very ad hoc. Witnessed the drawbacks of the pipelined boot- strapping paradigm, recent studies start turning to the end-to-end paradigm. For instance, Yan et al. (2020) propose the first end-to-end bootstrapping neural network for ESE, which uses the encoder- decoder architecture (see Figure 1b): the encoder leverages and encodes the co-occurrence relations between entities and patterns into their embeddings; the decoder models bootstrapping as a sequential entity generation process, and the generated enti- ties are used as expansion results. Compared with the pipeline paradigm, the end-to-end paradigm represents the whole bootstrapping process as a single learnable model and therefore is capable of leveraging more information and is more …

Content truncated to 500 words