Few-Shot Document-Level Relation Extraction

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5733 - 5746 July 10-15, 2022 ©2022 Association for Computational Linguistics Few-Shot Document-Level Relation Extraction Nicholas Popovic and Michael Färber Karlsruhe Institute of Technology (KIT), Germany {popovic, michael.faerber}@kit.edu Abstract We present FREDo, a few-shot document-level relation extraction (FSDLRE) benchmark. As opposed to existing benchmarks which are built on sentence-level relation extraction corpora, we argue that document-level corpora provide more realism, particularly regarding none-of- the-above (NOTA) distributions. Therefore, we propose a set of FSDLRE tasks and construct a benchmark based on two existing supervised learning data sets, DocRED and sciERC. We adapt the state-of-the-art sentence-level method MNAV to the document-level and develop it further for improved domain adaptation. We find FSDLRE to be a challenging setting with interesting new characteristics such as the abil- ity to sample NOTA instances from the support set. The data, code, and trained models are available online1. 1 Introduction The goal of relation extraction is to detect and classify relations between entities in a text accord- ing to a predefined schema. The schema, defining which relation types are relevant is highly depen- dent on the specific application and domain. Su- pervised learning methods for relation extraction (Soares et al., 2019; Zhou et al., 2021; Zhang et al., 2021; Xu et al., 2021; Xiao et al., 2022), which have advanced rapidly since the introduction of pretrained language models such as BERT (Devlin et al., 2019), need large corpora of annotated rela- tion instances to learn a schema. Since annotating data sets for relation extraction manually is expen- sive and time consuming, few-shot learning for relation extraction represents a promising solution for relation extraction at scale. While the general N-way K-shot few-shot learn- ing framework is relatively well defined and ap- pears easy to apply to relation extraction, construct- ing realistic benchmark tasks has proven to be chal- lenging. One of the core difficulties of establishing 1https://github.com/nicpopovic/FREDo a realistic benchmark task for few-shot relation ex- traction is correctly modelling the most frequent sit- uation a relation extraction system will encounter, none-of-the-above (NOTA) detection. NOTA refers to the case in which a candidate pair of entities does not hold any of the relations defined in the schema, a situation which is far more common than its re- verse (for the document-level data set DocRED (Yao et al., 2019), 96.84% of candidate entity pairs are NOTA cases). While initial benchmarks (Han et al., 2018) ignored this scenario altogether, re- searchers working on few-shot relation extraction have pushed for more realistic NOTA modeling in tasks and developed methods that can better de- tect NOTA instances (Gao et al., 2019; Sabo et al., 2021). Parallel to the outlined efforts towards realistic few-shot relation extraction benchmarks, research into supervised relation extraction has moved from sentence-level tasks, relation extraction within sin- gle sentences, to document-level relation extraction. The push towards document-level relation extrac- tion is motivated by (1) extracting more complex, cross-sentence relations and …

Content truncated to 500 words