Predicting sentential semantic compatibility for aggregation in text-to-text generation

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1061–1070, Osaka, Japan, December 11-17 2016. Predicting sentential semantic compatibility for aggregation in text-to-text generation Victor Chenal School of Computer Science McGill University victor.chenal@mail.mcgill.ca Jackie Chi Kit Cheung School of Computer Science McGill University jcheung@cs.mcgill.ca Abstract We examine the task of aggregation in the context of text-to-text generation. We introduce a new aggregation task which frames the process as grouping input sentence fragments into clusters that are to be expressed as a single output sentence. We extract datasets for this task from a cor- pus using an automatic extraction process. Based on the results of a user study, we develop two gold-standard clusterings and corresponding evaluation methods for each dataset. We present a hierarchical clustering framework for predicting aggregation decisions on this task, which out- performs several baselines and can serve as a reference in future work. 1 Introduction In text-to-text generation, existing sentence compression and sentence fusion systems assume that the in- put takes the form of one or more sentences, and the output is one suitably modified sentence (Barzilay et al., 1999; Knight and Marcu, 2000; Marsi and Krahmer, 2005; Filippova, 2010; Thadani and McKeown, 2013, for example). This line of work has led to new, knowledge-lean methods for sentence generation for applications such as text simplification and automatic summarization. The next step in expanding the scope of text-to-text generation is to relax assumptions about the forms of the input and output, with the eventual goal of generating entire passages or documents in an abstractive manner. We focus in this paper on aggregation, the task of determining what input units belong in the same output sentence. Aggregation is an important step in micro-planning in the traditional data-to-text NLG pipeline (Reiter et al., 2000). However, it has been little examined within the context of text-to-text generation outside of restricted syntactic contexts, such as entity-driven noun phrase rewriting (Nenkova, 2008). In this paper, we propose a new aggregation task suited for text-to-text generation. The goal of the task is to aggregate content into sentence-sized units by taking sentence fragments (i.e., meaningful bits of text) as input, and outputting clusters of these fragments. Figure 1 shows an example of such a clustering. It can be viewed as a preceding step for sentence fusion or other text-to-text generation methods. Inspired by surface realization tasks (Belz et al., 2011), we use pre-existing sentences as a source of data in order to cheaply and automatically generate datasets with different granularities. We conduct a user study to confirm the validity of these datasets and design two gold-standard clusterings. We use these gold standards to define two methods of evaluation, and introduce two baselines for the task. Then, we propose a simple clustering model for this task based on logistic regression and hierarchical clustering. All variants of the model are shown to outperform both baselines on our datasets, but an analysis of the results identifies shortcomings in the clustering that could be …

Content truncated to 500 words