Translation camera

MT Summit VII Sept. 1999 Translation Camera Yasuhiko Watanabe Yoshihiro Okada Ryukoku University Ryukoku University Seta, Otsu, Shiga, Japan Seta, Otsu, Shiga, Japan Yeun-Bae Kim Tetsuya Takeda NHK Science and Technical Research Labo. Ryukoku University Kinuta, Setagaya-ku, Tokyo, Japan Seta, Otsu, Shiga, Japan Abstract In this paper, we propose a camera system which translates Japanese texts in a scene. The system is portable and consists of four components: digital camera, character im- age extraction process, character recogni- tion process, and translation process. The system extracts character strings from a region which a user specifies, and trans- lates them into English. 1 Introduction There are many texts in various scenes: signboards, marks, indicators, and so on. Quite a lot of them give us important information: instructions, warnings, explanations, and so on. For example, characters in Figure 1 (a) give us an order, (tomare, stop)”. Natives have no problem to read them. However, it is not easy for foreigners to read them. Especially, it is more difficult when the characters are not their own. To solve this problem, we developed an experimen- tal system which we call translation camera. The sys- tem translates texts in scene images which are taken with a digital camera. In this paper, we describe the outline of our system and a method for recognizing and translating texts in scene images. 2 Texts in Scene Images 2.1 Related Work There are many attempts have been done on char- acter recognition in document images [Sakai 93]. On the contrary, there are only a few researches on charac- ter recognition in scene images [Choi 90] [Takizawa 95] [Ohya 94][Ueba 96]. It is because there are the follow- ing problems on character recognition in scene images: • it is not easy to obtain the clear contrast between character pattern strings and their background, • character pattern strings are often disturbed by noises, and • texts in scene images have a wide variety of the characters and the layouts. Moreover, the aims of the previous works are generally the automatic detection and extraction of character pattern strings in scene images. However, in the mo- bile application, a user has no difficulty in specifying a text in a scene image which he wants to translate. As a result, the aim and approach of our research are different from those of the previous works. 2.2 Important Texts in Scene Images Texts in scene images represent many kinds of infor- mation. Especially, the following kinds of information are important: • instruction (e.g. (tomare, stop)” in Fig- ure 1 (a)), • warning (e.g. (romen touketsu chui, mind the frozen road)” in Figure 1 (b)), • explanation (e.g. (Sanjo Keihan)" in Figure 1 (c)), and • name (e.g. (shokasen, fireplug)” in Fig- ure 1 (d)). Foreigners want to read texts which represent these kinds of information. As a result, in this study, we are concerned with the translation of these texts in scene images. It should be noted that even foreigners may dis- tinguish texts in scenes …

Content truncated to 500 words