Word Sense Acquisition from Bilingual Comparable Corpora

Word Sense Acquisition from Bilingual Comparable Corpora Hiroyuki Kaji Central Research Laboratory, Hitachi, Ltd. 1-280 Higashi-Koigakubo, Kokubunji-shi, Tokyo 185-8601, Japan kaji@crl.hitachi.co.jp Abstract Manually constructing an inventory of word senses has suffered from problems including high cost, arbitrary assignment of meaning to words, and mismatch to domains. To over- come these problems, we propose a method to assign word meaning from a bilingual comparable corpus and a bilingual dictionary. It clusters second-language translation equivalents of a first-language target word on the basis of their translingually aligned dis- tribution patterns. Thus it produces a hierar- chy of corpus-relevant meanings of the target word, each of which is defined with a set of translation equivalents. The effectiveness of the method has been demonstrated through an experiment using a comparable corpus con- sisting of Wall Street Journal and Nihon Kei- zai Shimbun corpora together with the EDR bilingual dictionary. 1 Introduction Word Sense Disambiguation (WSD) is an important subtask that is necessary for accomplishing most natu- ral language processing tasks including machine translation and information retrieval. A great deal of research on WSD has been done over the past decade (Ide and Veronis, 1998). In contrast, word sense acqui- sition has been a human activity; inventories of word senses have been constructed by lexicographers based on their intuition. Manually constructing an inventory of word senses has suffered from problems such as high cost, arbitrary division of word senses, and mis- match to application domains. We address the problem of word sense acquisition along the lines of the WSD where word senses are defined with sets of translation equivalents in another language. Bilingual corpora or second-language cor- pora enable unsupervised WSD (Brown, et al., 1991; Dagan and Itai, 1994). However, the correspondence between senses of a word and its translations is not one-to-one, and therefore we need to prepare an in- ventory of word senses, each of which is defined with a set of synonymous translation equivalents. Although conventional bilingual dictionaries usually group translations according to their senses, the grouping differs by dictionary. In addition, senses specific to a domain are often missing while many senses irrelevant to the domain or rare senses are included. To over- come these problems, we propose a method for pro- ducing a hierarchy of clusters of translation equiva- lents from a bilingual corpus and a bilingual diction- ary. To the best of our knowledge, there are two pre- ceding research papers on word sense acquisition (Fu- kumoto and Tsujii, 1994; Pantel and Lin, 2002). Both proposed distributional word clustering algorithms that are characterized by their capabilities to produce overlapping clusters. According to their algorithms, a polysemous word is assigned to multiple clusters, each of which represents one of its senses. These and our approach differ in how to define the word sense, i.e., a set of synonyms in the same language versus a set of translation equivalents in another language. Schuetze (1998) proposed a method for dividing occurrences of a word into classes, each of which consists of contex- …

Content truncated to 500 words