A Language Model-based Generative Classifier for Sentence-level Discourse Parsing

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2432–2446 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2432 A Language Model-based Generative Classifier for Sentence-level Discourse Parsing Ying Zhang, Hidetaka Kamigaito and Manabu Okumura Tokyo Institute of Technology {zhang,kamigaito,oku}@lr.pi.titech.ac.jp Abstract Discourse segmentation and sentence-level discourse parsing play important roles for vari- ous NLP tasks to consider textual coherence. Despite recent achievements in both tasks, there is still room for improvement due to the scarcity of labeled data. To solve the problem, we propose a language model-based genera- tive classifier (LMGC) for using more infor- mation from labels by treating the labels as an input while enhancing label representations by embedding descriptions for each label. More- over, since this enables LMGC to make ready the representations for labels, unseen in the pre-training step, we can effectively use a pre- trained language model in LMGC. Experimen- tal results on the RST-DT dataset show that our LMGC achieved the state-of-the-art F1 score of 96.72 in discourse segmentation. It further achieved the state-of-the-art relation F1 scores of 84.69 with gold EDU boundaries and 81.18 with automatically segmented boundaries, re- spectively, in sentence-level discourse parsing. 1 Introduction Textual coherence is essential for writing a natural language text that is comprehensible to readers. To recognize the coherent structure of a natural lan- guage text, Rhetorical Structure Theory (RST) is applied to describe an internal discourse structure for the text as a constituent tree (Mann and Thomp- son, 1988). A discourse tree in RST consists of elementary discourse units (EDUs), spans that de- scribe recursive connections between EDUs, and nuclearity and relation labels that describe relation- ships for each connection. Figure 1 (a) shows an example RST discourse tree. A span including one or more EDUs is a node of the tree. Given two adjacent non-overlapping spans, their nuclearity can be either nucleus or satellite, denoted by N and S, where the nucleus represents a more salient or essential piece of infor- mation than the satellite. Furthermore, a relation Weʼve got a lot to do , he acknowledged . Elaboration Attribution N S N S (a) Discourse Tree (Attribution (Elaboration (N (N We’ve got a lot )N (S to do , )S )N )Elaboration (S he acknowledged . )S )Attribution (b) Linearized Discourse Tree EDUs Figure 1: An example discourse tree structure. label, such as Attribution and Elaboration, is used to describe the relation between the given spans (Mann and Thompson, 1988; Carlson and Marcu, 2001). To build such trees, RST parsing consists of discourse segmentation, a task to detect EDU boundaries in a given text, and discourse parsing, a task to link spans for detected EDUs. In this paper, we focus on discourse segmenta- tion and sentence-level discourse parsing, which are indispensable in RST parsing (Joty et al., 2013; Feng and Hirst, 2014a; Joty et al., 2015; Wang et al., 2017; Kobayashi et al., 2020) and are appli- cable to many downstream tasks, such as machine translation (Guzmán et al., 2014; …

Content truncated to 500 words