One Vector is Not Enough: Entity-Augmented Distributed Semantics for Discourse Relations

One Vector is Not Enough: Entity-Augmented Distributed Semantics for Discourse Relations Yangfeng Ji and Jacob Eisenstein School of Interactive Computing Georgia Institute of Technology {jiyfeng, jacobe}@gatech.edu Abstract Discourse relations bind smaller linguistic units into coherent texts. Automatically identifying discourse relations is difficult, because it requires understanding the se- mantics of the linked arguments. A more subtle challenge is that it is not enough to represent the meaning of each argument of a discourse relation, because the rela- tion may depend on links between lower- level components, such as entity mentions. Our solution computes distributed mean- ing representations for each discourse ar- gument by composition up the syntactic parse tree. We also perform a downward compositional pass to capture the mean- ing of coreferent entity mentions. Implicit discourse relations are then predicted from these two representations, obtaining sub- stantial improvements on the Penn Dis- course Treebank. 1 Introduction The high-level organization of text can be char- acterized in terms of discourse relations between adjacent spans of text (Knott, 1996; Mann, 1984; Webber et al., 1999). Identifying these relations has been shown to be relevant to tasks such as summarization (Louis et al., 2010a; Yoshida et al., 2014), sentiment analysis (Somasundaran et al., 2009), coherence evaluation (Lin et al., 2011), and question answering (Jansen et al., 2014). While the Penn Discourse Treebank (PDTB) now pro- vides a large dataset annotated for discourse re- lations (Prasad et al., 2008), the automatic identi- fication of implicit relations is a difficult task, with state-of-the-art performance at roughly 40% (Lin et al., 2009). One reason for this poor performance is that dis- course relations are rooted in semantics (Forbes- u(ℓ) 0 Bob u(ℓ) 1 gave u(ℓ) 2 Tina u(ℓ) 3 the burger u(r) 0 She u(r) 1 was hungry (a) The distributed representations of burger and hungry are propagated up the parse tree, clarifying the implicit discourse relation between u(ℓ) 0 and u(r) 0 . d(ℓ) 0 Bob d(ℓ) 1 gave d(ℓ) 2 Tina u(ℓ) 3 the burger d(r) 0 She u(r) 1 was hungry (b) Distributed representations for the coreferent men- tions Tina and she are computed from the parent and sib- ling nodes. Figure 1: Distributed representations are com- puted through composition over the parse. Riley et al., 2006), which can be difficult to re- cover from surface level features. Consider the implicit discourse relation between the following two sentences (also shown in Figure 1a): (1) Bob gave Tina the burger. She was hungry. While a connector like because seems appropriate here, there is little surface information to signal this relationship, unless the model has managed to learn a bilexical relationship between burger and hungry. Learning all such relationships from an- notated data — including the relationship of hun- gry to knish, pierogie, pupusa etc — would require far more data than can possibly be annotated. 329 Transactions of the Association for Computational Linguistics, vol. 3, pp. 329–344, 2015. Action Editor: Alexander Koller. Submission batch: 12/2014; Revision batch 3/2015; Revision batch 4/2015; Published 6/2015. …

Content truncated to 500 words