Learning to Recognize Ancillary Information for Automatic Paraphrase Identification

Proceedings of NAACL-HLT 2016, pages 1109–1114, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Learning to Recognize Ancillary Information for Automatic Paraphrase Identification Simone Filice DICII, University of Roma, Tor Vergata filice@info.uniroma2.it Alessandro Moschitti∗ ALT, Qatar Computing Research Institute, HBKU amoschitti@qf.org.qa Abstract Previous work on Automatic Paraphrase Iden- tification (PI) is mainly based on modeling text similarity between two sentences. In con- trast, we study methods for automatically de- tecting whether a text fragment only appear- ing in a sentence of the evaluated sentence pair is important or ancillary information with re- spect to the paraphrase identification task. En- gineering features for this new task is rather difficult, thus, we approach the problem by representing text with syntactic structures and applying tree kernels on them. The results show that the accuracy of our automatic An- cillary Text Classifier (ATC) is promising, i.e., 68.6%, and its output can be used to improve the state of the art in PI. 1 Introduction Automatic PI is the task of detecting if two texts convey the same meaning. For example, the fol- lowing two sentences from the Microsoft Research Paraphrase Corpus (MSRP) (Dolan et al., 2004): S1a: Although it’s unclear whether Sobig was to blame, The New York Times also asked employees at its headquarters yesterday to shut down their com- puters because of ”system difficulties.” S1b: The New York Times asked employees at its headquarters to shut down their computers yester- day because of ”computing system difficulties.” are paraphrases, while these other two are not: ∗ Professor at DISI, University of Trento. S2a: Dr. Anthony Fauci, director of the National In- stitute of Allergy and Infectious Diseases, agreed. S2b: ”We have been somewhat lucky,” said Dr. An- thony Fauci, director of the National Institute of Al- lergy and Infectious Diseases. Most previous work on automatic PI, e.g., (Madnani et al., 2012; Socher et al., 2011), is based on a di- rect comparison between the two texts, exploiting different similarity scores into a machine learning framework. However, these methods consider sen- tences as monolithic units and can thus be misled by ancillary information that does not modify the main meaning expressed in the text. For example, the additional text fragment (ATF), “Although it’s unclear whether Sobig was to blame”, from S1a expresses ancillary information, which does not add much to the message of S1b, thus the sentences are considered paraphrases. In con- trast, S2b contains the ATF, “We have been some- what lucky”, whose meaning is not linked to any constituent of S1b. Since such text expresses rele- vant information, the two sentences are not consid- ered paraphrases. In this paper, we study and design models for ex- tracting ATFs from a sentence with respect to an- other one and classifying if their meaning is ancil- lary or important. For this purpose, we built a cor- pus of sentence pairs using MSRP, where at least one pair member always contains ATFs. We use SVMs with tree kernels applied to syntactic representations …

Content truncated to 500 words