Co-Occurrence Vectors From Corpora vs. Distance Vectors From Dictionaries

CO-OCCURRENCE VECTORS FROM CORPORA VS. DISTANCE VECTORS FROM DICTIONARIES Yoshiki Niwa ~tnd Yoshihiko Nitta Advanced Research l~aboratory, llitachi, l,td. Hatoyama, Saitam~t 350-03, .)ap;m {niwa2, nitta)~harl.hitachi.co.jp Abstract A comparison W~LS made of vectors derived by using ordinary co-occurrence statistics from large text cor- pora and of vectors derived by measuring the inter- word distances in dictionary definitions. The precision of word sense disambiguation by using co-occurrence vectors frorn the 1987 Wall Street Journal (20M total words) was higher than that by using distance vectors from the Collins English l)ictionary (60K head words + 1.6M definition words), llowever, other experimen-- tal results suggest that distance vectors contain some different semantic information from co-occurrence vec- tors. 1 Introduction Word vectors reflecting word meanings are expected to enable numerical approaches to semantics. Some early attempts at vector representation in I)sycholinguistics were the semantic d(O'erential approach (Osgood et al. 1957) and the associative distribution apl)roach (Deese 1962). llowever, they were derived manually through psychological experiments. An early attempt at automation was made I)y Wilks el aL (t990) us-. ing co-occurrence statistics. Since then, there haw" been some promising results from using co-occurrence vectors, such as word sense disambiguation (Schiitze [993), and word clustering (Pereira eL al. 1993). llowever, using the co-occurrence statistics re- quires a huge corpus that covers even most rare words. We recently developed word vectors that are derived from an ordinary dictionary by measuring the inter- word distances in the word definitions (Niwa and Nitta 1993). 'this method, by its nature, h~s no prol)lom handling rare words. In this paper we examine the nsefldness of these distance vectors as semantic re W resentations by comparing them with co-occur,'ence vectors. 2 Distance Vectors A reference network of the words in a dictionary (Fig. 1) is used to measure the distance between words, q'he network is a graph that shows which words are used in the. definition of each word (Nitta 1988). The network shown in Fig. 1 is for a w~ry small portion of the refer- ence network for the Collins English 1)ictionary (1979 edition) in the CI)-I{OM I (Liberman 1991), with 60K head words -b 1.6M definition words. writing unit (Or) \ / word comnmnieation / ~ alphMmtlcal \ L_\ / languag ,------ dictionary o, / \ ( : ) p,.~ople ~,ook (O~) Fig. 1. Portion of a reference network. For example, tile delinition for diclionarg is % book ill which the words of a language are listed al- phabetically .... " The word dicliona~d is thus linked to the words book, word, language, and alphabelical. A word w~etor is defined its the list of distances from a word to a certain sew of selected words, which we call origins. The words in Fig. 1 marked with Oi (unit, book, and people) m'e assumed to be origin words. In principle, origin words can be freoly chosen. In our exl~eriments we used mi(Idle fi'equency words: the 51st to 1050th most frequent words in the refer- ence Collins English I)ictiotmry (CI';D), The distance w~ctor …

Content truncated to 500 words