Joint Learning of Syntactic Features Helps Discourse Segmentation

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1073–1080 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1073 Joint Learning of Syntactic Features helps Discourse Segmentation Takshak Desai, Parag Dakle, Dan I. Moldovan Department of Computer Science The University of Texas at Dallas {takshak.desai, paragpravin.dakle, moldovan}@utdallas.edu Abstract This paper describes an accurate framework for carrying out multi-lingual discourse segmentation with BERT (Devlin et al., 2019). The model is trained to identify segments by casting the problem as a token classification problem and jointly learning syntactic features like part-of-speech tags and dependency relations. This leads to significant improvements in performance. Experiments are performed in different languages, such as English, Dutch, German, Portuguese Brazilian and Basque to highlight the cross-lingual effectiveness of the segmenter. In particular, the model achieves a state-of-the-art F-score of 96.7 for the RST-DT corpus (Carlson et al., 2003) improving on the previous best model by 7.2%. Additionally, a qualitative explanation is provided for how proposed changes contribute to model performance by analyzing errors made on the test data. Keywords: Discourse Segmentation; BERT; Multi-task learning 1. Introduction Discourse Segmentation refers to the task of fragmenting a document into minimal disjoint chunks of text called Ele- mentary Discourse Units (EDUs). In the context of Rhetor- ical Structure Theory (Mann and Thompson, 1988) or RST, EDUs form the nodes of a discourse tree; while relations between EDUs form arcs or edges between nodes. As a motivating example, consider the discourse tree given in Figure 1. EDUs labeled 1, 2 and 3 form nodes of the tree; and arcs are labeled with ATTRIBUTION (used to indicate instances of reported speech) and PURPOSE relations. This example was taken from the RST-DT corpus (Carlson et al., 2003) and the tree was constructed using the tool provided by Gessler et al. (2019). Figure 1: Discourse tree for a portion of text in wsj 2336: EDUs form the nodes of the tree, and arcs represents rela- tions. Discourse segmentation is considered a challenging prob- lem for several reasons. First, the boundary between syntax and discourse is blurry (Carlson and Marcu, 2001). While the clause is considered a basic EDU, segment boundaries are often determined using lexical and syntactic clues. Ad- ditionally, the amount of annotated data available is min- imal and this makes training of data-hungry models like neural networks difficult. To assuage the problem of data insufficiency, syntax-free models that leverage pre-trained representations (Wang et al., 2018; Muller et al., 2019) from sentence encoders like ElMo and BERT were proposed: these models achieved very good results for the segmentation task. Likewise, the use of syntactic features such as part-of-speech tags and parse tree features helped achieve better results (Braud et al., 2017; Lin et al., 2019). In fact, the latter achieved state-of-the-art results using pointer networks (Vinyals et al., 2015) with parse tree features. In this paper, we propose a few changes that leverage BERT’s (Devlin et al., 2019) structure in performing seg- mentation. Our …

Content truncated to 500 words