Lexical Choice via Topic Adaptation for Paraphrasing Written Language to Spoken Language

Lexical Choice via Topic Adaptation for Paraphrasing Written Language to Spoken Language Nobuhiro Kaji1 and Sadao Kurohashi2 1 Institute of Industrial Science, The University of Tokyo, 4-6-1 Komaba, Meguro-ku, Tokyo 153-8505, Japan kaji@tkl.iis.u-tokyo.ac.jp 2 Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan kuro@kc.t.u-tokyo.ac.jp Abstract. Our research aims at developing a system that paraphrases written language text to spoken language style. In such a system, it is important to distinguish between appropriate and inappropriate words in an input text for spoken language. We call this task lexical choice for paraphrasing. In this paper, we describe a method of lexical choice that considers the topic. Basically, our method is based on the word probabilities in written and spoken language corpora. The novelty of our method is topic adaptation. In our framework, the corpora are classified into topic categories, and the probability is estimated using such corpora that have the same topic as input text. The result of evaluation showed the effectiveness of topic adaptation. 1 Introduction Written language is different from spoken language. That difference has various aspects. For example, spoken language is often ungrammatical, or uses simplified words rather than difficult ones etc. Among these aspects this paper examines difficulty. Difficult words are characteristic of written language and are not ap- propriate for spoken language. Our research aims at developing a system that paraphrases written language text into spoken language style. It helps text-to-speech generating natural voice when the input is in written language. In order to create such a system, the following procedure is required: (1) the system has to detect inappropriate words in the input text for spoken language, (2) generate paraphrases of inappropriate words, and (3) confirm that the generated paraphrases are appropriate. This paper examines step (1) and (3), which we call lexical choice for paraphrasing written language to spoken language. Broadly speaking, lexical choice can be defined as binary classification task: the input is a word and a system outputs whether it is appropriate for spoken R. Dale et al. (Eds.): IJCNLP 2005, LNAI 3651, pp. 981–992, 2005. c ⃝ Springer-Verlag Berlin Heidelberg 2005 982 N. Kaji and S. Kurohashi language or not. This definition is valid if we can assume that the word diffi- culty is independent of such factors as context or listeners. However, we think such assumption is not always true. One example is business jargon (or technical term). Generally speaking, business jargon is difficult and inappropriate for spo- ken language. Notwithstanding, it is often used in business talk. This example implies that the word difficulty is dependent on the topic of text/talk. In this paper, we define the input of lexical choice as a word and text where it occurs (= the topic). Such definition makes it possible for a system to consider the topic. We think the topic plays an important role in lexical choice, when dealing with such words that are specific to a certain topic, e.g., business jargon. Hereafter, those …

Content truncated to 500 words