Extending Multi-Text Sentence Fusion Resources via Pyramid Annotations

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1854 - 1860 July 10-15, 2022 ©2022 Association for Computational Linguistics Extending Multi-Text Sentence Fusion Resources via Pyramid Annotations Daniela Brook Weiss 1 Paul Roit 1 Ori Ernst 1 Ido Dagan1 1Computer Science Department, Bar-Ilan University {dani.b.weiss,plroit,oriern}@gmail.com dagan@cs.biu.ac.il Abstract NLP models that process multiple texts of- ten struggle in recognizing corresponding and salient information that is often differently phrased, and consolidating the redundancies across texts. To facilitate research of such challenges, the sentence fusion task was pro- posed, yet previous datasets for this task were very limited in their size and scope. In this paper, we revisit and substantially extend pre- vious dataset creation efforts. With careful modifications, relabeling and employing com- plementing data sources, we were able to more than triple the size of a notable ear- lier dataset. Moreover, we show that our ex- tended version uses more representative texts for multi-document tasks and provides a more diverse training-set, which substantially im- proves model performance. 1 Introduction Despite recent advances reported in NLU bench- marks for single document tasks, cross-document tasks, such as multi-document summarization (MDS) have not progressed with the same pace. The handling of information across documents re- quires effective measures for identifying overlap- ping content. Moreover, generative tasks require consolidating the relevant and redundant content into a coherent utterance. In light of this, sev- eral works proposed a focused sentence-level task, called sentence fusion, which focuses on summa- rizing multiple sentences with overlapping con- tent into a non-redundant one. This allows a fine- grained analysis of which information units are shared among the input sentences, as well as con- trol over different degrees of information inclusion and exclusion. However, the available datasets for fusing sen- tences which exhibit significant content overlap are still lacking, with the most recent containing only several hundreds of examples (McKeown et al., a. Fisheries in parts of the Philippines have been deci- mated by the use of cyanide in fishing. b. Philippine fishermen use cyanide in fishing, needlessly destroying immature fish. c. Sodium cyanide use by fisherman decimates fish. d. In the Philippines some fishermen use homemade explo- sives and cyanide for driving fish away from reefs and into nets. Label Sodium cyanide use by fisherman decimates fish Table 1: Sentence fusion example from Thadani and McKeown (2013). (a-d) are the input sentences, origi- nating from different documents. Text spans (in bold) that are considered as contributing to the same unit of content (SCU) are annotated with the same concise label. The sentences where the spans appear in are grouped to be input for sentence fusion, while the SCU label becomes the fusion target. 2010; Thadani and McKeown, 2013), impeding further research. In this work, we follow Thadani and McKeown (2013) and extend their described sentence fusion dataset, which is derived from ex- pertly written and annotated summaries based on the Pyramid MDS evaluation method by Nenkova and Passonneau (2004). …

Content truncated to 500 words