Modeling Extractive Sentence Intersection via Subtree Entailment

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2891–2901, Osaka, Japan, December 11-17 2016. Modeling Extractive Sentence Intersection via Subtree Entailment Omer Levy∗ Ido Dagan∗ Gabriel Stanovsky∗ Judith Eckle-Kohler† Iryna Gurevych†‡ ∗Computer Science Department, Bar-Ilan University, Israel †Ubiquitous Knowledge Processing Lab, Technische Universit¨at Darmstadt, Germany ‡Ubiquitous Knowledge Processing Lab, German Institute for Educational Research, Germany Abstract Sentence intersection captures the semantic overlap of two texts, generalizing over paradigms such as textual entailment and semantic text similarity. Despite its modeling power, it has re- ceived little attention because it is difficult for non-experts to annotate. We analyze 200 pairs of similar sentences and identify several underlying properties of sentence intersection. We leverage these insights to design an algorithm that decomposes the sentence intersection task into several simpler annotation tasks, facilitating the construction of a high quality dataset via crowdsourcing. We implement this approach and provide an annotated dataset of 1,764 sentence intersections. 1 Introduction Various paradigms exist for comparing the meanings of two texts and modeling their semantic overlap. Paraphrase detection (Dolan et al., 2004), for example, tries to identify whether two texts express the same information. It cannot, however, capture cases where there is only partial information overlap. One paradigm that addresses this issue is textual entailment (Dagan et al., 2006), which asks whether one text can be inferred from another. While textual entailment models when the information of one text is subsumed by another, it falls short when neither text strictly subsumes the other, yet a significant amount of information is common to both. A more recent paradigm, semantic text similarity (Agirre et al., 2012), allows for these cases by measuring “how much” similar two sentences are. In this paper, we consider an additional paradigm for modeling the semantic overlap between two texts. The task of sentence intersection, as defined in several variants (Marsi and Krahmer, 2005; McKeown et al., 2010; Thadani and McKeown, 2011), is to construct a sentence containing all the information shared by the two input sentences (Figure 1). While sentence intersection was originally proposed for abstractive summarization, we argue that it is an interesting generic paradigm for modeling information overlap, which generalizes over several previous approaches. In particular, it is expressive enough to capture partial semantic overlaps that are not modeled by textual entailment or even partial textual entailment (Nielsen et al., 2009; Levy et al., 2013). Furthermore, rather than quantifying the amount of shared information as in semantic text similarity, sentence intersection captures what this shared information is. Although sentence intersection has existed for over a decade, it has received little attention due to a lack of annotated data. Previous annotation attempts have either used experts, which did not scale, or crowdsourcing, which yielded unreliable annotations (McKeown et al., 2010). We also observe that an- notating sentence intersection is difficult for non-experts. We hypothesize that this difficulty stems from the task’s requirement that the annotator indicate all the shared information when constructing the inter- section, necessitating a high …

Content truncated to 500 words