Bringing the Dictionary to the User: The FOKS System

Bringing the Dictionary to the User: the FOKS system Slaven Bilac†, Timothy Baldwin∗ and Hozumi Tanaka† † Tokyo Institute of Technology 2-12-1 Ookayama, Meguro-ku, Tokyo 152-8552 JAPAN {sbilac,tanaka}@cl.cs.titech.ac.jp ∗ CSLI, Ventura Hall, Stanford University Stanford, CA 94305-4115 USA tbaldwin@csli.stanford.edu Abstract The dictionary look-up of unknown words is partic- ularly difficult in Japanese due to the complicated writing system. We propose a system which allows learners of Japanese to look up words according to their expected, but not necessarily correct, reading. This is an improvement over previous systems which provide no handling of incorrect readings. In prepro- cessing, we calculate the possible readings each kanji character can take and different types of phonolog- ical and conjugational changes that can occur, and associate a probability with each. Using these prob- abilities and corpus-based frequencies we calculate a plausibility measure for each generated reading given a dictionary entry, based on the naive Bayes model. In response to a reading input, we calculate the plau- sibility of each dictionary entry corresponding to the reading and display a list of candidates for the user to choose from. We have implemented our system in a web-based environment and are currently eval- uating its usefulness to learners of Japanese. 1 Introduction Unknown words are a major bottleneck for learners of any language, due to the high overhead involved in looking them up in a dictionary. This is particularly true in non-alphabetic languages such as Japanese, as there is no easy way of looking up the component characters of new words. This research attempts to alleviate the dictionary look-up bottleneck by way of a comprehensive dictionary interface which allows Japanese learners to look up Japanese words in an ef- ficient, robust manner. While the proposed method is directly transferable to other language pairs, for the purposes of this paper, we will focus exclusively on a Japanese–English dictionary interface. The Japanese writing system consists of the three orthographies of hiragana, katakana and kanji, which appear intermingled in modern-day texts (NLI, 1986). The hiragana and katakana syllabaries, collectively referred to as kana, are relatively small (46 characters each), and each character takes a unique and mutually exclusive reading which can easily be memorized. Thus they do not present a major difficulty for the learner. Kanji characters (ideograms), on the other hand, present a much big- ger obstacle. The high number of these characters (1,945 prescribed by the government for daily use, and up to 3,000 appearing in newspapers and formal publications) in itself presents a challenge, but the matter is further complicated by the fact that each character can and often does take on several different and frequently unrelated readings. The kanji � , for example, has readings including hatsu and ta(tsu), whereas � has readings including omote, hyou and arawa(reru). Based on simple combinatorics, there- fore, the kanji compound � � happyou “announce- ment” can take at least 6 basic readings, and when one considers phonological and conjugational varia- tion, this number becomes much greater. Learners presented with the …

Content truncated to 500 words