HistRED: A Historical Document-Level Relation Extraction Dataset

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3207–3224 July 9-14, 2023 ©2023 Association for Computational Linguistics HistRED: A Historical Document-Level Relation Extraction Dataset Soyoung Yang Minseok Choi Youngwoo Cho Jaegul Choo KAIST AI {sy_yang, minseok.choi, cyw314, jchoo}@kaist.ac.kr Abstract Despite the extensive applications of relation extraction (RE) tasks in various domains, lit- tle has been explored in the historical context, which contains promising data across hundreds and thousands of years. To promote the his- torical RE research, we present HistRED con- structed from Yeonhaengnok. Yeonhaengnok is a collection of records originally written in Hanja, the classical Chinese writing, which has later been translated into Korean. HistRED pro- vides bilingual annotations such that RE can be performed on Korean and Hanja texts. In addi- tion, HistRED supports various self-contained subtexts with different lengths, from a sentence level to a document level, supporting diverse context settings for researchers to evaluate the robustness of their RE models. To demonstrate the usefulness of our dataset, we propose a bilingual RE model that leverages both Ko- rean and Hanja contexts to predict relations be- tween entities. Our model outperforms mono- lingual baselines on HistRED, showing that employing multiple language contexts supple- ments the RE predictions. The dataset is pub- licly available at: https://huggingface.co/ datasets/Soyoung/HistRED under CC BY- NC-ND 4.0 license. 1 Introduction Relation extraction (RE) is the task of extracting re- lational facts from natural language texts. To solve RE problems, diverse datasets and machine learn- ing (ML) methods have been developed. Earlier work limits the scope of the problem to sentence- level RE, in which the task is to predict a relation- ship between two entities in a single sentence (Dod- dington et al., 2004; Walker et al., 2006; Hendrickx et al., 2010; Alt et al., 2020; Stoica et al., 2021). However, such a setting is impractical in real-world applications where relations between entities can exist across sentences in large unstructured texts. Therefore, document-level RE datasets for general and biomedical domains have been introduced (Li [1] 잠시 앉아서 응대했다. [2] 관할하는 여러 시의 낭관이 일제히 와서 인사를 드렸다. [3] 경영 관사에 이르러 관복을 정제하고 가니, 부사ㆍ서장관 및 각 부서의 원역들이 무리를 따라 나갔다. [4] 돈의문으로 들어가, 종루를 지났다. [5] 눈에 띄는 고향의 모습이 전이나 다름없었다. [6] 지난해 8월에 작별하던 때의 회포를 돌이켜 생각하니, 눈물이 하염없이 솟는다. Korean [1] 所按諸寺郞官. [2] 齊進投刺. [3] 進到京營官舍. [4]整冠服以行. [5] 副价行臺曁各務員役. [6] 逐隊以進. [7] 入敦義門. [8] 過鐘樓故里. [9] 物色觸目如舊. [10] 回思仲秋別時情懷. Hanja I sat down for a while to respond. Many of the city’s officials came at once to greet me. When I reached the management office and refined my official clothes, BusaㆍSeoJangKwon and the officers of each department followed the crowd. Entering Donuimun Gate, I passed a bell tower. The appearance of my hometown was same as before. When I looked back on the recollection of saying goodbye in August last year, my tears well up. English* Entity: Person, Location, Clothes, Datetime Relation {sbj_kor: “경영 관사”, obj_kor: “돈의문”, sbj_han: …

Content truncated to 500 words