Analysis of Machine Translation Systems’ Errors in Tense, Aspect, and Modality

Proceedings of PACLIC 19, the 19th Asia-Pacific Conference on Language, Information and Computation. Analysis of Machine Translation Systems' Errors in Tense, Aspect, and Modality Masaki Murata National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Soraku-gun Kyoto, 619-0289, Japan murata@nict.go.jp Kiyotaka Uchimoto National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Soraku-gun Kyoto, 619-0289, Japan uchimoto@nict.go.jp Qing Ma Ryukoku University Otsu, Shiga, 520-2194, Japan qma@math.ryukoku.ac.jp Toshiyuki Kanamaru Kyoto University Yoshida-Nihonmatsu-cho, Sakyo-ku Kyoto, 606-8501, Japan kanamaru@hi.h.kyoto-u.ac.jp Hitoshi Isahara National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Soraku-gun Kyoto, 619-0289, Japan isahara@nict.go.jp Abstract Errors of the translation of tense, aspect, and modality by machine translation systems were analyzed for six translation systems on the market and our new systems for translating tense, aspect, and modality. The results showed that our systems outperformed the other systems. They also showed that the other systems often produced progressive forms rather than the correct present forms. Our systems rarely made this mistake. Translation systems on the market could thus be improved by incorporating the methods used in our systems. Moreover, error analysis of the translation systems on the market identified information that would be useful for improving them. 1. Introduction Tense, aspect, and modality are difficult to translate appropriately using machines (Shirai et al., 1990; Kume et al., 1990; Dale et al., 2000; Nirenburg et al., 2002). We investigated the error patterns produced by translation systems when translating Japanese tense, aspect, and modality expressions into English. We compared the performance of six translation systems on the market and our new translation systems for tense, aspect, and modality. We found that our systems outperformed the other systems, and we detected error patterns that the other systems often made and our systems rarely made. These results can be used to improve translation systems on the market. Moreover, we extracted error patterns peculiar to each translation system. These errors can be corrected easily because their corresponding sentences can be translated correctly by other systems. These results are useful for improving each translation system. 2. Method In our investigation, we considered that the translation of tense, aspect, and modality from Japanese to English means the production of the surface expressions of tense, aspect, and modality of the main verb phrase in the English translated sentence. We calculated the accuracy rates and extracted the error patterns in the translations. We used combinations of the following categories as the surface expressions of tense, aspect, and modality. We refer to the categories as the categories of tense, aspect, and modality. 1. all combinations of {present tense, past tense}, {progressive, non-progressive}, and {perfect, non-perfect} (8 categories) 2. imperative mood (1 category) 3. auxiliary verbs ({present tense, past tense} of “be able to”, {present tense, past tense} of “be going to”, {present tense, past tense} of “be to”, “can”, “could”, and {present tense, past tense} of “have to”, “had better”, “may”, “might”, “must”, “need”, “ought”, “shall”, “should”, “used to”, “will”, and “would”) (21 categories) 4. noun phrases (one category) 5. participial …

Content truncated to 500 words