Unsupervised Sense Disambiguation Using Bilingual Probabilistic Models

Unsupervised Sense Disambiguation Using Bilingual Probabilistic Models Indrajit Bhattacharya Dept. of Computer Science University of Maryland College Park, MD, USA indrajit@cs.umd.edu Lise Getoor Dept. of Computer Science University of Maryland College Park, MD, USA getoor@cs.umd.edu Yoshua Bengio Dept. IRO Universit´e de Montr´eal Montr´eal, Qu´ebec, Canada bengioy@IRO.UMontreal.CA Abstract We describe two probabilistic models for unsuper- vised word-sense disambiguation using parallel cor- pora. The first model, which we call the Sense model, builds on the work of Diab and Resnik (2002) that uses both parallel text and a sense in- ventory for the target language, and recasts their ap- proach in a probabilistic framework. The second model, which we call the Concept model, is a hier- archical model that uses a concept latent variable to relate different language specific sense labels. We show that both models improve performance on the word sense disambiguation task over previous unsu- pervised approaches, with the Concept model show- ing the largest improvement. Furthermore, in learn- ing the Concept model, as a by-product, we learn a sense inventory for the parallel language. 1 Introduction Word sense disambiguation (WSD) has been a cen- tral question in the computational linguistics com- munity since its inception. WSD is fundamental to natural language understanding and is a useful in- termediate step for many other language process- ing tasks (Ide and Veronis, 1998). Many recent approaches make use of ideas from statistical ma- chine learning; the availability of shared sense defi- nitions (e.g. WordNet (Fellbaum, 1998)) and recent international competitions (Kilgarrif and Rosen- zweig, 2000) have enabled researchers to compare their results. Supervised approaches which make use of a small hand-labeled training set (Bruce and Wiebe, 1994; Yarowsky, 1993) typically out- perform unsupervised approaches (Agirre et al., 2000; Litkowski, 2000; Lin, 2000; Resnik, 1997; Yarowsky, 1992; Yarowsky, 1995), but tend to be tuned to a specific corpus and are constrained by scarcity of labeled data. In an effort to overcome the difficulty of find- ing sense-labeled training data, researchers have be- gun investigating unsupervised approaches to word- sense disambiguation. For example, the use of par- allel corpora for sense tagging can help with word sense disambiguation (Brown et al., 1991; Dagan, 1991; Dagan and Itai, 1994; Ide, 2000; Resnik and Yarowsky, 1999). As an illustration of sense disam- biguation from translation data, when the English word bank is translated to Spanish as orilla, it is clear that we are referring to the shore sense of bank, rather than the financial institution sense. The main inspiration for our work is Diab and Resnik (2002), who use translations and linguistic knowledge for disambiguation and automatic sense tagging. Bengio and Kermorvant (2003) present a graphical model that is an attempt to formalize probabilistically the main ideas in Diab and Resnik (2002). They assume the same semantic hierarchy (in particular, WordNet) for both the languages and assign English words as well as their translations to WordNet synsets. Here we present two variants of the graphical model in Bengio and Kermorvant (2003), along with a method to …

Content truncated to 500 words