A Generic Sentence Trimmer with CRFs

Proceedings of ACL-08: HLT, pages 299–307, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics A Generic Sentence Trimmer with CRFs Tadashi Nomoto National Institute of Japanese Literature 10-3, Midori Tachikawa Tokyo, 190-0014, Japan nomoto@acm.org Abstract The paper presents a novel sentence trimmer in Japanese, which combines a non-statistical yet generic tree generation model and Con- ditional Random Fields (CRFs), to address improving the grammaticality of compres- sion while retaining its relevance. Experi- ments found that the present approach out- performs in grammaticality and in relevance a dependency-centric approach (Oguro et al., 2000; Morooka et al., 2004; Yamagata et al., 2006; Fukutomi et al., 2007) − the only line of work in prior literature (on Japanese compres- sion) we are aware of that allows replication and permits a direct comparison. 1 Introduction For better or worse, much of prior work on sentence compression (Riezler et al., 2003; McDonald, 2006; Turner and Charniak, 2005) turned to a single cor- pus developed by Knight and Marcu (2002) (K&M, henceforth) for evaluating their approaches. The K&M corpus is a moderately sized corpus consisting of 1,087 pairs of sentence and compres- sion, which account for about 2% of a Ziff-Davis collection from which it was derived. Despite its limited scale, prior work in sentence compression relied heavily on this particular corpus for establish- ing results (Turner and Charniak, 2005; McDonald, 2006; Clarke and Lapata, 2006; Galley and McKe- own, 2007). It was not until recently that researchers started to turn attention to an alternative approach which does not require supervised data (Turner and Charniak, 2005). Our approach is broadly in line with prior work (Jing, 2000; Dorr et al., 2003; Riezler et al., 2003; Clarke and Lapata, 2006), in that we make use of some form of syntactic knowledge to constrain com- pressions we generate. What sets this work apart from them, however, is a novel use we make of Conditional Random Fields (CRFs) to select among possible compressions (Lafferty et al., 2001; Sut- ton and McCallum, 2006). An obvious benefit of using CRFs for sentence compression is that the model provides a general (and principled) proba- bilistic framework which permits information from various sources to be integrated towards compress- ing sentence, a property K&M do not share. Nonetheless, there is some cost that comes with the straightforward use of CRFs as a discriminative classifier in sentence compression; its outputs are often ungrammatical and it allows no control over the length of compression they generates (Nomoto, 2007). We tackle the issues by harnessing CRFs with what we might call dependency truncation, whose goal is to restrict CRFs to working with can- didates that conform to the grammar. Thus, unlike McDonald (2006), Clarke and Lap- ata (2006) and Cohn and Lapata (2007), we do not insist on finding a globally optimal solution in the space of 2n possible compressions for an n word long sentence. Rather we insist on finding a most plausible compression among those that are explic- itly warranted by the grammar. …

Content truncated to 500 words