Sentence Compression with Joint Structural Inference

Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pages 65–74, Sofia, Bulgaria, August 8-9 2013. c⃝2013 Association for Computational Linguistics Sentence Compression with Joint Structural Inference Kapil Thadani and Kathleen McKeown Department of Computer Science Columbia University New York, NY 10025, USA {kapil,kathy}@cs.columbia.edu Abstract Sentence compression techniques often assemble output sentences using frag- ments of lexical sequences such as n- grams or units of syntactic structure such as edges from a dependency tree repre- sentation. We present a novel approach for discriminative sentence compression that unifies these notions and jointly pro- duces sequential and syntactic represen- tations for output text, leveraging a com- pact integer linear programming formula- tion to maintain structural integrity. Our supervised models permit rich features over heterogeneous linguistic structures and generalize over previous state-of-the- art approaches. Experiments on corpora featuring human-generated compressions demonstrate a 13-15% relative gain in 4- gram accuracy over a well-studied lan- guage model-based compression system. 1 Introduction Recent years have seen increasing interest in text- to-text generation tasks such as paraphrasing and text simplification, due in large part to their direct utility in high-level natural language tasks such as abstractive summarization. The task of sentence compression in particular has benefited from the availability of a number of useful resources such as the the Ziff-Davis compression corpus (Knight and Marcu, 2000) and the Edinburgh compression corpus (Clarke and Lapata, 2006b) which make compression problems highly relevant for data- driven approaches involving language generation. The sentence compression task addresses the problem of minimizing the lexical footprint of a sentence, i.e., the number of words or characters in it, while preserving its most salient informa- tion. This is illustrated in the following example from the compression corpus of Clarke and Lap- ata (2006b): Original: In 1967 Chapman, who had cultivated a conventional image with his ubiquitous tweed jacket and pipe, by his own later admission stunned a party attended by his friends and future Python colleagues by coming out as a homosexual. Compressed: In 1967 Chapman, who had cultivated a conventional image, stunned a party by coming out as a homosexual. Compression can therefore be viewed as analo- gous to text summarization1 defined at the sen- tence level. Unsurprisingly, independent selec- tion of tokens for an output sentence does not lead to fluent or meaningful compressions; thus, compression systems often assemble output text from units that are larger than single tokens such as n-grams (McDonald, 2006; Clarke and Lap- ata, 2008) or edges in a dependency structure (Fil- ippova and Strube, 2008; Galanis and Androut- sopoulos, 2010). These systems implicitly rely on a structural representation of text—as a sequence of tokens or as a dependency tree respectively—to to underpin the generation of an output sentence. In this work, we present structured transduc- tion: a novel supervised framework for sen- tence compression which employs a joint infer- ence strategy to simultaneously recover sentence compressions under both these structural repre- sentations of text—a token sequence as well as a tree of syntactic dependencies. Sentence genera- …

Content truncated to 500 words