Dialogue-Based Relation Extraction

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4927–4940 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4927 Dialogue-Based Relation Extraction Dian Yu1† Kai Sun2† Claire Cardie2 Dong Yu1 1Tencent AI Lab, Bellevue, WA 2Cornell University, Ithaca, NY {yudian, dyu}@tencent.com, ks985@cornell.edu, cardie@cs.cornell.edu Abstract We present the first human-annotated dialogue- based relation extraction (RE) dataset Dialo- gRE, aiming to support the prediction of re- lation(s) between two arguments that appear in a dialogue. We further offer DialogRE as a platform for studying cross-sentence RE as most facts span multiple sentences. We ar- gue that speaker-related information plays a critical role in the proposed task, based on an analysis of similarities and differences be- tween dialogue-based and traditional RE tasks. Considering the timeliness of communication in a dialogue, we design a new metric to evaluate the performance of RE methods in a conversational setting and investigate the per- formance of several representative RE meth- ods on DialogRE. Experimental results demon- strate that a speaker-aware extension on the best-performing model leads to gains in both the standard and conversational evaluation set- tings. DialogRE is available at https:// dataset.org/dialogre/. 1 Introduction Cross-sentence relation extraction, which aims to identify relations between two arguments that are not mentioned in the same sentence or relations that cannot be supported by any single sentence, is an essential step in building knowledge bases from large-scale corpora automatically (Ji et al., 2010; Swampillai and Stevenson, 2010; Surdeanu, 2013). It has yet to receive extensive study in nat- ural language processing, however. In particular, although dialogues readily exhibit cross-sentence relations, most existing relation extraction tasks fo- cus on texts from formal genres such as profession- ally written and edited news reports or well-edited websites (Elsahar et al., 2018; Yao et al., 2019; † Equal contribution. S1: Hey Pheebs. S2: Hey! S1: Any sign of your brother? S2: No, but he’s always late. S1: I thought you only met him once? S2: Yeah, I did. I think it sounds y’know big sistery, y’know, ‘Frank’s always late.’ S1: Well relax, he’ll be here. Argument pair Trigger Relation type R1 (Frank, S2) brother per:siblings R2 (S2, Frank) brother per:siblings R3 (S2, Pheebs) none per:alternate names R4 (S1, Pheebs) none unanswerable Table 1: A dialogue and its associated instances in Di- alogRE. S1, S2: anoymized speaker of each utterance. Mesquita et al., 2019; Grishman, 2019), while dia- logues have been under-studied. In this paper, we take an initial step towards studying relation extraction in dialogues by con- structing the first human-annotated dialogue-based relation extraction dataset, DialogRE. Specifically, we annotate all occurrences of 36 possible relation types that exist between pairs of arguments in the 1,788 dialogues originating from the complete tran- scripts of Friends, a corpus that has been widely employed in dialogue research in recent years (Cati- zone et al., 2010; Chen and Choi, 2016; Chen et al., 2017; Zhou and Choi, 2018; Rashid and Blanco, 2018; Yang and Choi, 2019). Altogether, we an- notate 10,168 relational …

Content truncated to 500 words