Distributed Marker Representation for Ambiguous Discourse Markers and Entangled Relations

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5334–5351 July 9-14, 2023 ©2023 Association for Computational Linguistics Distributed Marker Representation for Ambiguous Discourse Markers and Entangled Relations Dongyu Ru1, Lin Qiu1, Xipeng Qiu2, Yue Zhang3, Zheng Zhang1 1Amazon AWS AI 2School of Computer Science, Fudan University 3School of Engineering, Westlake University {rudongyu,quln,zhaz}@amazon.com xpqiu@fudan.edu.cn zhangyue@westlake.edu.cn Abstract Discourse analysis is an important task because it models intrinsic semantic structures between sentences in a document. Discourse markers are natural representations of discourse in our daily language. One challenge is that the mark- ers as well as pre-defined and human-labeled discourse relations can be ambiguous when describing the semantics between sentences. We believe that a better approach is to use a contextual-dependent distribution over the markers to express discourse information. In this work, we propose to learn a Distributed Marker Representation (DMR) by utilizing the (potentially) unlimited discourse marker data with a latent discourse sense, thereby bridg- ing markers with sentence pairs. Such repre- sentations can be learned automatically from data without supervision, and in turn provide insights into the data itself. Experiments show the SOTA performance of our DMR on the im- plicit discourse relation recognition task and strong interpretability. Our method also offers a valuable tool to understand complex ambigu- ity and entanglement among discourse markers and manually defined discourse relations. 1 Introduction Discourse analysis is a fundamental problem in natural language processing. It studies the linguis- tic structures beyond the sentence boundary and is a component of chains of thinking. Such struc- tural information has been widely applied in many downstream applications, including information extraction (Peng et al., 2017), long documents sum- marization (Cohan et al., 2018), document-level machine translation (Chen et al., 2020), conversa- tional machine reading (Gao et al., 2020). Discourse relation recognition (DRR) focuses on semantic relations, namely, discourse senses between sentences or clauses. Such inter-sentence structures are sometimes explicitly expressed in nat- ural language by discourse connectives, or markers Figure 1: Entangled discourse relations and correspond- ing markers between clauses. As shown in the figure, there exist diverse discourse relations (marked in blue) and corresponding markers (marked in red) for the same pair of clauses. It suggests that the semantic meaning of different discourse relations can be entangled to each other. (e.g., and, but, or). The availability of these mark- ers makes it easier to identify corresponding rela- tions (Pitler et al., 2008), as is in the task of explicit discourse relation recognition (EDRR), since there is strong correlation between discourse markers and relations. On the contrary, implicit discourse relation recognition (IDRR), where markers are missing, remains a more challenging problem. Prior work aims to address such challenges by making use of discourse marker information over explicit data in learning implicit discourse rela- tions, either by injecting marker prediction knowl- edge into a representation model (Zhou et al., 2010; Braud and Denis, 2016), or transferring the marker prediction task into implicit discourse relation pre- diction by manually defining …

Content truncated to 500 words