Flexible Japanese Sentence Compression by Relaxing Unit Constraints

Proceedings of COLING 2012: Technical Papers, pages 1097–1112, COLING 2012, Mumbai, December 2012. Flexible Japanese Sentence Compression by Relaxing Unit Constraints Jun Harashima Sadao Kurohashi Graduate School of Informatics, Kyoto University Yoshida-honmachi, Sakyo-ku, Kyoto, 606-8501, Japan harashima@nlp.ist.i.kyoto-u.ac.jp, kuro@i.kyoto-u.ac.jp ABSTRACT Sentence compression is important in a wide range of applications in natural language process- ing. Previous approaches of Japanese sentence compression can be divided into two groups. Word-based methods extract a subset of words from a sentence to shorten it, while bunsetsu- based methods extract a subset of bunsetsu (where a bunsetsu is a text unit that consists of content words and following function words). Basically, bunsetsu-based methods perform bet- ter than word-based methods. However, bunsetsu-based methods have the disadvantage that they cannot drop unimportant words from each bunsetsu because they have to follow con- straints under which each bunsetsu is treated as a unit. In this paper, we propose a novel compression method to overcome this disadvantage. Our method relaxes the constraints using Lagrangian relaxation and shortens each bunsetsu if it contains unimportant words. Exper- imental results show that our method effectively compresses a sentence while preserving its important information and grammaticality. TITLE AND ABSTRACT IN JAPANESE ユニット制約の緩和による柔軟な日本語文圧縮 文圧縮は，自然言語処理の様々なアプリケーションにおいて重要である．日本語文に対する 既存の圧縮手法は二種類に分けられる．単語ベースの手法は文から単語集合を選出し，圧縮 文とする．一方，文節ベースの手法は文から文節集合を選出し，圧縮文とする．基本的には 後者の方が良く機能する．しかし，文節ベースの手法は，文節をユニットとして扱うという 制約があるため，個々の文節を圧縮できない．本稿では，この欠点を克服する新しい圧縮手 法を提案する．提案手法はラグランジュ緩和を用いて上の制約を緩和し，各文節を圧縮する． 実験の結果，提案手法によって原文の情報を多く保持する文法的な圧縮文を生成できること が分かった． KEYWORDS: sentence compression, Lagrangian relaxation. KEYWORDS IN JAPANESE: 文圧縮, ラグランジュ緩和. 1097 1 Introduction Sentence compression is the task of shortening a sentence while preserving its important in- formation and grammaticality. This task is important in a wide range of applications such as automatic summarization (Jing, 2000; Lin, 2003; Zajic et al., 2007), subtitle generation (Vandeghinste and Pan, 2004), and displaying text on small screens (Corston-Oliver, 2001). In this paper, we propose a novel compression method for a Japanese sentence. Like other languages, Japanese uses sentences composed of words. However, we can also say that a Japanese sentence is composed of bunsetsu. Bunsetsu is a text unit that consists of one or more content words and possibly one or more function words. For example, consider the following sentence.1 (1) nihon Japan to CNJ kanada Canada no GEN kokusai international kyoudou collaborative kenkyuu research guru-pu group ga NOM hakken discover shita did (An international collaborative research group between Japan and Canada made a dis- covery) This sentence is composed of four bunsetsu: “nihon to”, “kanada no”, “kokusai kyoudou kenkyuu guru-pu ga”, and “hakken shita”. As seen in this example, a Japanese sentence can be viewed as a bunsetsu sequence as well as a word sequence. This characteristic of the Japanese language has led researchers to take two compression approaches: word-based methods and bunsetsu-based methods. Word-based methods view a source sentence as a word sequence and generate a compressed sentence by selecting a sub- set of words from the source sentence (Hori and Furui, 2004; Hirao et al., 2009). However, the methods do not take account of bunsetsu, and it is thus difficult to generate grammatical compressions. For example, if only content words (or only function words) in a bunsetsu are selected, the grammaticality of the corresponding …

Content truncated to 500 words