Unsupervised Discourse Constituency Parsing Using Viterbi EM

Unsupervised Discourse Constituency Parsing Using Viterbi EM Noriki Nishida and Hideki Nakayama Graduate School of Information Science and Technology The University of Tokyo {nishida,nakayama}@nlab.ci.i.u-tokyo.ac.jp Abstract In this paper, we introduce an unsupervised discourse constituency parsing algorithm. We use Viterbi EM with a margin-based crite- rion to train a span-based discourse parser in an unsupervised manner. We also propose initialization methods for Viterbi training of discourse constituents based on our prior knowledge of text structures. Experimental re- sults demonstrate that our unsupervised parser achieves comparable or even superior perfor- mance to fully supervised parsers. We also investigate discourse constituents that are learned by our method. 1 Introduction Naturallanguagetextisgenerallycoherent(Halliday and Hasan, 1976) and can be analyzed as discourse structures, which formally describe how text is coherently organized. In discourse structure, lin- guistic units (e.g., clauses, sentences, or larger textual spans) are connected together semantically and pragmatically, and no unit is independent nor isolated. Discourse parsing aims to uncover dis- course structures automatically for given text and has been proven to be useful in various NLP applications, such as document summarization (Marcu, 2000; Louis et al., 2010; Yoshida et al., 2014), sentiment analysis (Polanyi and Van den Berg, 2011; Bhatia et al., 2015), and automated essay scoring (Miltsakaki and Kukich, 2004). Despite the promising progress achieved in re- cent decades (Carlson et al., 2001; Hernault et al., 2010; Ji and Eisenstein, 2014; Feng and Hirst, 2014; Li et al., 2014; Joty et al., 2015; Morey et al., 2017), discourse parsing still remains a significant challenge. The difficulty is due in part to shortage and low reliability of hand-annotated discourse structures. To develop a better-generalized parser, existing algorithms require a larger amounts of training data. However, manually annotating dis- course structures is expensive, time-consuming, and sometimes highly ambiguous (Marcu et al., 1999). One possible solution to these problems is gram- mar induction (or unsupervised syntactic parsing) algorithms for discourse parsing. However, exist- ing studies on unsupervised parsing mainly focus on sentence structures, such as phrase structures (Lari and Young, 1990; Klein and Manning, 2002; Golland et al., 2012; Jin et al., 2018) or dependency structures (Klein and Manning, 2004; Berg-Kirkpatrick et al., 2010; Naseem et al., 2010; Jiang et al., 2016), though text-level structural reg- ularities can also exist beyond the scope of a single sentence. For instance, in order to convey infor- mation to readers as intended, a writer should ar- range utterances in a coherent order. We tackle these problems by introducing unsu- pervised discourse parsing, which induces dis- course structures for given text without relying on human-annotated discourse structures. Based on Rhetorical Structure Theory (RST) (Mann and Thompson, 1988), which is one of the most widely accepted theories of discourse structure, we assume that coherent text can be represented as tree structures, such as the one in Figure 1. The leaf nodes correspond to non-overlapping clause- level text spans called elementary discourse units (EDUs). Consecutive text spans are combined to each other recursively in a bottom–up manner to form larger text …

Content truncated to 500 words