Ontology-Style Relation Annotation: A Case Study

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4867–4876 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4867 Ontology-Style Relation Annotation: A Case Study Savong Bou, Naoki Suzuki, Makoto Miwa and Yutaka Sasaki Computational Intelligence (COIN) Lab, Toyota Technological Institute {savong, sd16042, makoto-miwa, yutaka.sasaki}@toyota-ti.ac.jp Abstract This paper proposes an Ontology-Style Relation (OSR) annotation approach. In conventional Relation Extraction (RE) datasets, relations are annotated as links between entity mentions. In contrast, in our OSR annotation, a relation is annotated as a relation mention (i.e., not a link but a node) and domain and range links are annotated from the relation mention to its argument entity mentions. We expect the following benefits: (1) the relation annotations can be easily converted to Resource Description Framework (RDF) triples to populate an Ontology, (2) some part of conventional RE tasks can be tackled as Named Entity Recognition (NER) tasks. The relation classes are limited to several RDF properties such as domain, range, and subClassOf, and (3) OSR annotations can be clear documentations of Ontology contents. As a case study, we converted an in-house corpus of Japanese traffic rules in conventional annotations into the OSR annotations and built a novel OSR-RoR (Rules of the Road) corpus. The inter-annotator agreements of the conversion were 85-87%. We evaluated the performance of neural NER and RE tools on the conventional and OSR annotations. The experimental results showed that the OSR annotations make the RE task easier while introducing slight complexity into the NER task. Keywords: Ontology-style relation annotation, relation extraction, named entity recognition, corpus construction 1. Introduction Relation Extraction (RE) is the task to find predefined re- lations between target entity terms in a text. Traditionally, RE studies rely on corpora that have term annotations and relation link annotations between two terms. In conventional relation annotations, relations are anno- tated as links between target entity mentions. In contrast, in our Ontology-Style Relation (OSR) annotations, a rela- tion is annotated as a relation mention (i.e., not a link but a node) and then domain and range links are annotated from the relation mention to its entity mentions. Similar to the Ontology structures, the domain link connects the relation mention to the source target entity while the range link con- nects the relation mention to the destination entity. By tak- ing this approach, we expect the following benefits: • Since the relation annotations can be easily converted to Ontology RDF (Resource Description Framework)1 triples, the annotated relations can be used to populate Ontology entries. • Because relations are annotated as relation mentions, some part of the relation-type classification task on a conventional RE corpus become a Named Entity Recognition (NER) task, in which deep learning is quite effective to achieve over 80% F-scores for many NE types (Mai et al., 2018), compared to less investi- gated RE task with many relation types, e.g., (Zhang et al., 2017). It is also worth addressing that the number of Ontology relation classes to be …

Content truncated to 500 words