Automatic Prediction of Discourse Connectives

Automatic Prediction of Discourse Connectives Eric Malmi1,*, Daniele Pighin2, Sebastian Krause2,*, Mikhail Kozhevnikov2 1Aalto University 2Google Espoo, Finland Z¨urich, Switzerland eric.malmi@aalto.fi {biondo,bastik,qnan}@google.com Abstract Accurate prediction of suitable discourse connectives (however, furthermore, etc.) is a key component of any system aimed at building coherent and fluent discourses from shorter sentences and passages. As an example, a dialog system might assemble a long and informative answer by sampling passages extracted from different documents retrieved from the Web. We formulate the task of discourse connective prediction and release a dataset of 2.9M sentence pairs separated by discourse connectives for this task. Then, we evaluate the hardness of the task for human raters, apply a recently proposed decomposable attention (DA) model to this task and observe that the automatic predictor has a higher F1 than human raters (32 vs. 30). Nevertheless, under specific conditions the raters still outperform the DA model, suggesting that there is headroom for future improvements. Keywords: discourse connectives, decomposable attention model, discourse relation prediction 1. Introduction Discourse connectives, also referred to as discourse mark- ers, discourse cues, or discourse adverbials, are used to bind together and to explicate the relation between pieces of text. It is a common language class exercise to be asked to fill in suitable connectives to a text in order to improve the text flow. Similarly, it is important for computational summa- rization and text adaptation systems to be able to fill in suit- able discourse connectives to produce natural-sounding ut- terances. In this work, we study the problem of automatic discourse connective prediction. We limit ourselves to connectives which appear at the beginning of a sentence, linking the sentence to the preceding one. Even in this limited set- ting, an automatic discourse connective predictor has many concrete use cases. For example, in a question-answering setting it could help to generate answers by collating sen- tences from multiple sources. In extractive text summariza- tion, it could be used to determine what is the best way to join two sentences that used to be separated by one or more sentences. As part of a text-authoring application, it could suggest suitable connectives at the beginning of a sentence. In the literature, discourse connective prediction has been recently studied merely as an intermediate step for the well-studied problem of implicit discourse relation predic- tion (Xu et al., 2012; Zhou et al., 2010). However, consider- ing the aforementioned applications, we argue that connec- tive prediction makes an interesting and relevant problem in its own right. The contributions of this work are twofold: 1. We present an extensive experimental study on the problem of discourse connective prediction and show that a recently proposed decomposable attention model (Parikh et al., 2016) yields a good performance on this task. The model clearly outperforms a popu- lar word-pair model and obtains a better performance than human raters on the same task and data. 2. We describe the dataset that we collected, consist- *Work performed during an internship at Google. ing of 2.9 million adjacent …

Content truncated to 500 words