Measuring and Predicting Orthographic Associations: Modelling the Similarity of Japanese Kanji

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 1041–1048 Manchester, August 2008 Measuring and Predicting Orthographic Associations: Modelling the Similarity of Japanese Kanji Lars Yencken and Timothy Baldwin {lljy,tim}@csse.unimelb.edu.au NICTA Research Lab University of Melbourne Abstract As human beings, our mental processes for recognising linguistic symbols gen- erate perceptual neighbourhoods around such symbols where confusion errors oc- cur. Such neighbourhoods also pro- vide us with conscious mental associa- tions between symbols. This paper for- malises orthographic models for similarity of Japanese kanji, and provides a proof- of-concept dictionary extension leveraging the mental associations provided by ortho- graphic proximity. 1 Introduction Electronic dictionary interfaces have evolved from mere digitised forms of their paper ancestors. They now enhance accessibility by addressing the sep- arate needs of language consumers and language producers, of learners from non-speakers to native speakers, and by targeting the specific difficulties presented by individual languages. For languages with logographic orthographies, such as Japanese and Chinese, accessibility re- mains poor due to the difficulties in looking up an unknown character in the dictionary. The tra- ditional method of character lookup in these lan- guages involves identifying the primary compo- nent (or “radical”), counting its strokes, looking it up in the index, counting the remainder of strokes in the original character, then finding the character in a sub-index. This presents several opportunities for error, but fortunately improvements have been made, as we discuss in Section 2. We are interested in the perceptual process of identifying characters, in particular the behaviour of perception within dense visual neighbourhoods. Within the dictionary accessibility space, we are © 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported license (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. motivated by the potential to correct confusion er- rors, but also to leverage the mental associations provided by visual proximity to allow advanced learners to find unknown characters faster. As proof-of-concept, we propose a method for look- ing up unknown words with unfamiliar characters, based on similarity with known characters. In essence, our method is based on the user plausibly “mistyping” the word based on closely- matching kanji they are familiar with (and hence can readily access via a standard input method ed- itor), from which we predict the correct kanji com- bination based on kanji similarity and word fre- quency. For example, given the input 補左, the system could suggest the word 補佐 [hosa] “help” based on similarity between the high-frequency 左 and the graphically-similar but low-frequency 佐. The proposed method is combined with the FOKS lookup strategy proposed by Bilac (2002) for looking up unknown words via plausibly incor- rect readings. The contributions of this paper are the proposal of a range of character similarity models for logo- graphic scripts, a novel evaluation method for lo- gographic character confusability, and the incorpo- ration of kanji similarity into a word-level lookup model. The remainder of this paper is structured as fol- lows. Firstly, we review related lookup systems (Section 2), and go on to discuss how we …

Content truncated to 500 words