Automatic Comma Insertion for Japanese Text Generation

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 892–901, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Automatic Comma Insertion for Japanese Text Generation Masaki Murata Graduate School of Information Science, Nagoya University, Japan murata@el.itc.nagoya-u.ac.jp Tomohiro Ohno Graduate School of International Development, Nagoya University, Japan ohno@nagoya-u.jp Shigeki Matsubara Graduate School of Information Science, Nagoya University, Japan matubara@nagoya-u.jp Abstract This paper proposes a method for automat- ically inserting commas into Japanese texts. In Japanese sentences, commas play an im- portant role in explicitly separating the con- stituents, such as words and phrases, of a sen- tence. The method can be used as an ele- mental technology for natural language gen- eration such as speech recognition and ma- chine translation, or in writing-support tools for non-native speakers. We categorized the usages of commas and investigated the ap- pearance tendency of each category. In this method, the positions where commas should be inserted are decided based on a machine learning approach. We conducted a comma insertion experiment using a text corpus and confirmed the effectiveness of our method. 1 Introduction In Japanese sentences, commas are inserted to mark word boundaries that might be otherwise unclear be- cause Japanese is a non-segmented language. They are also inserted at sharp semantic boundaries to im- prove the readability of a sentence. While there is a tendency about the positions where commas should be inserted in a Japanese sentence, there is no clear standard for these positions. Therefore, it is hard for non-natives of Japanese such as foreign students to insert commas properly, and the method for au- tomatic comma insertion is required to support sen- tence generation by such people. In addition, this method is expected to be useful for improving read- ability of texts generated by automatic speech recog- nition or machine translation. This paper proposes a method for automatically inserting commas into Japanese texts. There are several usages of commas, and the positions to in- sert commas depend on these usages. Therefore, we grouped the usages of commas into nine cate- gories, and investigated the appearance tendency for each category to find the effective features of ma- chine learning by using Japanese newspaper arti- cles. Based on the analysis of comma positions, our method decides whether or not to insert a comma at each bunsetsu1 boundary in an input sentence by machine learning. We conducted an experiment on comma insertion using the Kyoto Text Corpus (Kurohashi and Nagao, 1998), and obtained higher recall and precision than those of the baseline, leading us to confirm the ef- fectiveness of our method. This paper is organized as follows: The next sec- tion presents related works. Section 3 gives prelim- inary analyses. Section 4 explains how our comma insertion method works. An experiment and discus- sions are presented in Sections 5 and 6, respectively. 2 Related Works There have been many investigations on comma in- sertion into output texts of speech recognition sys- tems to improve the readability (Christensen …

Content truncated to 500 words