From Zero to Hero: Human-In-The-Loop Entity Linking in Low Resource Domains

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6982–6993 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6982 From Zero to Hero: Human-In-The-Loop Entity Linking in Low Resource Domains Jan-Christoph Klie Richard Eckart de Castilho Iryna Gurevych Ubiquitous Knowledge Processing Lab (UKP-TUDA) Department of Computer Science Technical University of Darmstadt, Germany www.ukp.tu-darmstadt.de Abstract Entity linking (EL) is concerned with disam- biguating entity mentions in a text against knowledge bases (KB). It is crucial in a consid- erable number of fields like humanities, tech- nical writing and biomedical sciences to en- rich texts with semantics and discover more knowledge. The use of EL in such domains requires handling noisy texts, low resource set- tings and domain-specific KBs. Existing ap- proaches are mostly inappropriate for this, as they depend on training data. However, in the above scenario, there exists hardly annotated data, and it needs to be created from scratch. We therefore present a novel domain-agnostic Human-In-The-Loop annotation approach: we use recommenders that suggest potential con- cepts and adaptive candidate ranking, thereby speeding up the overall annotation process and making it less tedious for users. We evaluate our ranking approach in a simulation on diffi- cult texts and show that it greatly outperforms a strong baseline in ranking accuracy. In a user study, the annotation speed improves by 35 % compared to annotating without interactive support; users report that they strongly prefer our system. An open-source and ready-to-use implementation based on the text annotation platform INCEpTION1 is made available2. 1 Introduction Entity linking (EL) describes the task of disam- biguating entity mentions in a text by linking them to a knowledge base (KB), e.g. the text span Earl of Orrery can be linked to the KB entry John Boyle, 5. Earl of Cork, thereby disambiguating it. EL is highly beneficial in many fields like digital hu- manities, classics, technical writing or biomedical sciences for applications like search (Meij et al., 1https://inception-project.github.io 2https://github.com/UKPLab/ acl2020-interactive-entity-linking Figure 1: Difficult entity mentions with their linked en- tities: 1) Name variations, 2) Spelling Variation, 3) Am- biguity 2014), semantic enrichment (Schl¨ogl and Lejtovicz, 2017) or information extraction (Nooralahzadeh and Øvrelid, 2018). These are overwhelmingly low-resource settings: often, no data annotated ex- ists; coverage of open-domain knowledge bases like Wikipedia or DBPedia is low. Therefore, en- tity linking is frequently performed against domain- specific knowledge bases (Munnelly and Lawless, 2018a; Bartsch, 2004). In these scenarios, the first crucial step is to ob- tain annotated data. This data can then be either directly used by researchers for their downstream task or to train machine learning models for au- tomatic annotation. For this initial data creation step, we developed a novel Human-In-The-Loop (HITL) annotation approach. Manual annotation is laborious and often prohibitively expensive. To improve annotation speed and quality, we there- fore add interactive machine learning annotation support that helps the user find entities in the text and select the correct knowledge base entries for them. The more entities are annotated, the better …

Content truncated to 500 words