Evaluation Dataset and Methodology for Extracting Application-Specific Taxonomies from the Wikipedia Knowledge Graph

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2341–2347 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2341 Evaluation Dataset and Methodology for Extracting Application-Specific Taxonomies from the Wikipedia Knowledge Graph Georgeta Bordea1, Stefano Faralli2, Fleur Mougin1, Paul Buitelaar3, and Gayo Diallo1 1 University of Bordeaux, Inserm, Bordeaux Population Health Research Center, ERIAS team, France 2University of Rome Unitelma Sapienza, Italy 3Data Science Institute, NUI Galway, Ireland {georgeta.bordea, fleur.mougin, gayo.diallo}@u-bordeaux.fr, stefano.faralli@unitelmasapienza.it, paul.buitelaar@nuigalway.ie Abstract In this work, we address the task of extracting application-specific taxonomies from the category hierarchy of Wikipedia. Previous work on pruning the Wikipedia knowledge graph relied on silver standard taxonomies which can only be automatically extracted for a small subset of domains rooted in relatively focused nodes, placed at an intermediate level in the knowledge graphs. In this work, we propose an iterative methodology to extract an application-specific gold standard dataset from a knowledge graph and an evaluation framework to comparatively assess the quality of noisy automatically extracted taxonomies. We employ an existing state-of-the-art algorithm in an iterative manner and we propose several sampling strategies to reduce the amount of manual work needed for evaluation. A first gold standard dataset is released to the research community for this task along with a companion evaluation framework. This dataset addresses a real-world application from the medical domain, namely the extraction of food-drug and herb-drug interactions. Keywords: Taxonomy Extraction, Knowledge Graphs, Knowledge Graph Pruning 1. Introduction Taxonomies are useful tools for content organisation, nav- igation, and information retrieval, providing valuable input for semantically intensive tasks such as question answering (Ojokoh and Adebisi, 2019) and textual entailment (Fawei et al., 2019). Recent efforts on setting common grounds for taxonomy extraction evaluation in the context of SemEval have focused on extracting hierarchical relations from un- structured text (Bordea et al., 2015; Bordea et al., 2016; Jurgens and Pilehvar, 2016). In the past few years, this task attracted an increasing amount of interest from the research community. But the relatively low evaluation results show that this is still a challenging task, despite efforts to sim- plify the task by focusing on the more simple subtask of hy- pernym relations extraction rather than constructing a full fledged taxonomy and by providing datasets that mainly cover concepts with a relatively high frequency in the target corpus (Camacho-Collados et al., 2018). Instead of extract- ing hierarchical relations from text alone, in this work, we address the problem of adapting and re-using existing taxo- nomical structures from general-purpose knowledge graphs such as the Wikipedia knowledge graph (Kapanipathi et al., 2014). The sheer vastness of Wikipedia’s domain coverage recom- mends it as a reliable source of semi-structured informa- tion. Manually-curated and high-quality information about hierarchical relations is readily available for a wide range of domains. Hence, the taxonomy extraction task should be made more feasible provided that there are methods to accurately identify relevant information for a given domain or application scenario. The challenge is to deal with …

Content truncated to 500 words