Discriminative Sentence Compression with Soft Syntactic Evidence

Discriminative Sentence Compression with Soft Syntactic Evidence Ryan McDonald Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104 ryantm@cis.upenn.edu Abstract We present a model for sentence com- pression that uses a discriminative large- margin learning framework coupled with a novel feature set defined on compressed bigrams as well as deep syntactic repre- sentations provided by auxiliary depen- dency and phrase-structure parsers. The parsers are trained out-of-domain and con- tain a significant amount of noise. We ar- gue that the discriminative nature of the learning algorithm allows the model to learn weights relative to any noise in the feature set to optimize compression ac- curacy directly. This differs from cur- rent state-of-the-art models (Knight and Marcu, 2000) that treat noisy parse trees, for both compressed and uncompressed sentences, as gold standard when calculat- ing model parameters. 1 Introduction The ability to compress sentences grammatically with minimal information loss is an important problem in text summarization. Most summariza- tion systems are evaluated on the amount of rele- vant information retained as well as their compres- sion rate. Thus, returning highly compressed, yet informative, sentences allows summarization sys- tems to return larger sets of sentences and increase the overall amount of information extracted. We focus on the particular instantiation of sen- tence compression when the goal is to produce the compressed version solely by removing words or phrases from the original, which is the most com- mon setting in the literature (Knight and Marcu, 2000; Riezler et al., 2003; Turner and Charniak, 2005). In this framework, the goal is to find the shortest substring of the original sentence that con- veys the most important aspects of the meaning. We will work in a supervised learning setting and assume as input a training set T =(xt, yt)|T | t=1 of original sentences xt and their compressions yt. We use the Ziff-Davis corpus, which is a set of 1087 pairs of sentence/compression pairs. Fur- thermore, we use the same 32 testing examples from Knight and Marcu (2000) and the rest for training, except that we hold out 20 sentences for the purpose of development. A handful of sen- tences occur twice but with different compres- sions. We randomly select a single compression for each unique sentence in order to create an un- ambiguous training set. Examples from this data set are given in Figure 1. Formally, sentence compression aims to shorten a sentence x = x1 . . . xn into a substring y = y1 . . . ym, where yi ∈ {x1, . . . , xn}. We define the function I(yi) ∈ {1, . . . , n} that maps word yi in the compression to the index of the word in the original sentence. Finally we include the con- straint I(yi) < I(yi+1), which forces each word in x to occur at most once in the compression y. Compressions are evaluated on three criteria, 1. Grammaticality: Compressed sentences should be grammatical. 2. Importance: How much of the important …

Content truncated to 500 words