An effective Discourse Parser that uses Rich Linguistic Information

Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the ACL, pages 566–574, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics An effective Discourse Parser that uses Rich Linguistic Information Rajen Subba ∗ Display Advertising Sciences Yahoo! Labs Sunnyvale, CA, USA rajen@yahoo-inc.com Barbara Di Eugenio Department of Computer Science University of Illinois Chicago, IL, USA bdieugen@cs.uic.edu Abstract This paper presents a first-order logic learn- ing approach to determine rhetorical relations between discourse segments. Beyond lin- guistic cues and lexical information, our ap- proach exploits compositional semantics and segment discourse structure data. We report a statistically significant improvement in clas- sifying relations over attribute-value learn- ing paradigms such as Decision Trees, RIP- PER and Naive Bayes. For discourse pars- ing, our modified shift-reduce parsing model that uses our relation classifier significantly outperforms a right-branching majority-class baseline. 1 Introduction Many theories postulate a hierarchical structure for discourse (Mann and Thompson, 1988; Moser et. al., 1996; Polanyi et. al., 2004). Discourse struc- ture is most often based on semantic / pragmatic re- lationships between spans of text and results in a tree structure, as that shown in Figure 1. Discourse parsing, namely, deriving such tree structures and the rhetorical relations labeling their inner nodes is still a challenging and mostly unsolved problem in NLP. It is linguistically plausible that such structures are determined at least in part on the basis of the meaning of the related chunks of texts, and of the rhetorical intentions of their authors. However, such knowledge is extremely difficult to capture. Hence, previous work on discourse parsing (Wellner et. al., 2006; Sporleder and Lascarides, 2005; Marcu, 2000; Polanyi et. al., 2004; Soricut and Marcu, 2003; ∗This work was done while the author was a student at the University of Illinois at Chicago. Baldridge and Lascarides, 2005) has relied only on syntactic and lexical information, lexical chains and shallow semantics. We present an innovative discourse parser that uses compositional semantics (when available) and information on the structure of the segment being built itself. Our discourse parser, based on a modi- fied shift-reduce algorithm, crucially uses a rhetori- cal relation classifier to determine the site of attach- ment of a new incoming chunk together with the ap- propriate relation label. Another novel aspect of our work is the usage of Inductive Logic Programming (ILP): ILP learns from first-order logic representa- tions (FOL). The ILP-based relation classifier is significantly more accurate than relation classifiers that use competitive propositional ML algorithms such as decision trees and Naive Bayes. In addi- tion, it results in FOL rules that are linguistically perspicuous. Our domain is that of instructional how-to-do manuals, and we describe our corpus in Section 2. In Section 3, we discuss the modified shift-reduce parser we developed. The bulk of the paper is devoted to the rhetorical relation classifier in Section 4. Experimental results of both the rela- tion classifier and the discourse parser in its entirety are discussed in Section 5. Further details can be found in …

Content truncated to 500 words