Weak Supervision for Learning Discourse Structure

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2296–2305, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2296 Weak Supervision for Learning Discourse Structure Sonia Badene1,2, Kate Thompson1,2, Jean-Pierre Lorr´e2, Nicholas Asher1,3,4 1IRIT, 2Linagora, 3CNRS, 4ANITI {sonia.badene,kate.thompson,nicholas.asher}@irit.fr, {sbadene,jplorre}@linagora.com Abstract This paper provides a detailed comparison of a data programming approach with (i) off-the-shelf, state-of-the-art deep learning architectures that optimize their representa- tions (BERT) and (ii) handcrafted-feature ap- proaches previously used in the discourse analysis literature. We compare these ap- proaches on the task of learning discourse structure for multi-party dialogue. The data programming paradigm offered by the Snorkel framework allows a user to label training data using expert-composed heuristics, which are then transformed via the “generative step” into probability distributions of the class labels given the data. We show that on our task the generative model outperforms both deep learning architectures as well as more tradi- tional ML approaches when learning discourse structure—it even outperforms the combina- tion of deep learning methods and hand- crafted features. We also implement several strategies for “decoding” our generative model output in order to improve our results. We conclude that weak supervision methods hold great promise as a means for creating and im- proving data sets for discourse structure. 1 Introduction In this paper, we investigate and demonstrate the potential of a weak supervision, data program- ming approach (Ratner et al., 2016) to the task of learning discourse structure for multi-party di- alogue. We offer a detailed comparison of our data programming approach with (i) off-the-shelf, state-of-the-art deep learning architectures that optimize their representations (BERT) and (ii) handcrafted-feature approaches previously used in the discourse analysis literature. Our data pro- gramming paradigm exploits the Snorkel frame- work that allows a user to label training data us- ing expert-composed heuristics, which are then transformed via the “generative step” into prob- ability distributions of the class labels given the data. We show that the generative model produced from these heuristics outperforms both deep learn- ing architectures as well as more traditional ML approaches when learning discourse structure by up to 20 points of F1 score; it even outperforms the combination of generative and discriminative approaches that are the foundation of the Snorkel framework. We also implement several strategies for “decoding” our generative model output that improve our results. We assume discourse structures are dependency structures (Muller et al., 2012; Li et al., 2014) and restrict the structure learning problem to predict- ing edges or attachments between discourse unit (DU) pairs in the dependency graph. Although the problem of attachment is only a part of the over- all task of discourse interpretation, it is a difficult problem that serves as a useful benchmark for var- ious approaches to discourse parsing. After train- ing a supervised deep learning algorithm to pre- dict attachments on the STAC annotated corpus1, we then constructed a weakly supervised learning system in which we used 10% …

Content truncated to 500 words