T-REx: A Large Scale Alignment of Natural Language with Knowledge Base Triples

T-REx: A Large Scale Alignment of Natural Language with Knowledge Base Triples Hady Elsahar∗, Pavlos Vougiouklis†, Arslen Remaci∗, Christophe Gravier∗, Jonathon Hare†, Elena Simperl†, Frederique Laforest∗ ∗Universit´e de Lyon, Laboratoire Hubert Curien, Saint-´Etienne, France †Electronics and Computer Science, University of Southampton, UK {hady.elsahar, christophe.gravier, frederique.laforest}@univ-st-etienne.fr arslen.remaci@etu.univ-st-etienne.fr {pv1e13, jsh2, e.simperl}@ecs.soton.ac.uk Abstract Alignments between natural language and Knowledge Base (KB) triples are an essential prerequisite for training machine learning approaches employed in a variety of Natural Language Processing problems. These include Relation Extraction, KB Population, Question Answering and Natural Language Generation from KB triples. Available datasets that provide those alignments are plagued by significant shortcomings – they are of limited size, they exhibit a restricted predicate coverage, and/or they are of unreported quality. To alleviate these shortcomings, we present T-REx, a dataset of large scale alignments between Wikipedia abstracts and Wikidata triples. T-REx consists of 11 million triples aligned with 3.09 million Wikipedia abstracts (6.2 million sentences). T-REx is two orders of magnitude larger than the largest available alignments dataset and covers 2.5 times more predicates. Additionally, we stress the quality of this language resource thanks to an extensive crowdsourcing evaluation. T-REx is publicly available at https://w3id.org/t-rex. Keywords: Knowledge Base Population, Relation Extraction, Distant Supervision, Wikidata 1. Introduction Reducing the gap between Natural Language and struc- tured knowledge bases (KB) has been the concern of many research tasks such as: Relation Extraction [Mintz et al.2009], KB Population [Ji and Grishman2011], KB- driven Natural Language Generation [Lebret et al.2016] and Question Answering [Xu et al.2016]. Models built for these tasks rely on training datasets containing align- ments between sentences in free text and KB triples. Pre- vious works [Mintz et al.2009,Yao et al.2011] have created alignments either manually or automatically for the pur- pose of training and evaluating their models. [Augenstein et al.2016, Martin et al.2016] have pointed out shortcomings of existing alignments language resources, and showed the importance of building a dataset of high quality large scale alignments. Such shortcomings include: 1) their limited size in terms of the number of alignments, 2) their lim- ited coverage as the number of represented predicates is not enough to generalize to larger domains (usually such datasets are very biased towards few predicates), and/or 3) their either low or unreported quality. In this work, we build T-REx, a large scale alignment dataset between free text documents and KB triples. T- REx consists of 3.09 million Wikipedia abstracts aligned with 11 million Wikidata triples, covering more than 600 unique Wikidata predicates. T-REx is two orders of mag- nitude larger than the largest available alignments dataset, and covers 2.5 times more predicates. In this paper, we de- fine the customizable architecture of the alignment pipeline which uses three different automatic alignment techniques. We evaluate the quality of T-REx by running a crowdsourc- ing experiment over 2,600 created alignments. The best au- tomatic alignment technique in T-REx achieved an accuracy of 97.8% over the evaluated subset of the dataset. 2. Related Work A considerable body of work has …

Content truncated to 500 words