Using ‘smart’ bilingual projection to feature-tag a monolingual dictionary

Using ‘smart’ bilingual projection to feature-tag a monolingual dictionary Katharina Probst Language Technologies Institute School of Computer Science Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 kathrin@cs.cmu.edu Abstract We describe an approach to tagging a monolin- gual dictionary with linguistic features. In par- ticular, we annotate the dictionary entries with parts of speech, number, and tense information. The algorithm uses a bilingual corpus as well as a statistical lexicon to find candidate train- ing examples for specific feature values (e.g. plural). Then a similarity measure in the space defined by the training data serves to define a classifier for unseen data. We report evaluation results for a French dictionary, while the ap- proach is general enough to be applied to any language pair. In a further step, we show that the proposed framework can be used to assign linguistic roles to extracted morphemes, e.g. noun plu- ral markers. While the morphemes can be extracted using any algorithm, we present a simple algorithm for doing so. The emphasis hereby is not on the algorithm itself, but on the power of the framework to assign roles, which are ultimately indispensable for tasks such as Machine Translation. 1 Introduction and motivation The Machine Translation community has recently under- gone a major shift of focus towards data-driven tech- niques. Among these techniques, example-based (e.g. (Brown, 1997)) and statistical (e.g. (Brown et al., 1990; Brown et al., 1993)) are best known and stud- ied. They aim at extracting information from bilingual text and building translation systems automatically. This empirical approach overcomes the development bottle- neck that traditional transfer- and interlingua-based ap- proaches face. What used to take years of human devel- opment time can now be achieved in a fraction of the time with similar accuracy. However, in studying such empir- ical approaches and the output of the resulting systems, there have been calls for the re-incorporation of more lin- guistic intuition and/or knowledge. One notable example in this context is (Yamada and Knight, 2001; Yamada and Knight, 2002), who introduce syntactic knowledge into their statistical translation model. Our approach goes in a similar direction. The AVENUE system (Carbonell et al., 2002) infers syntactic transfer rules similar to the ones a human grammar writer would produce. The training data is bilingual text, and learning is facilitated by the usage of linguistic information (e.g. parses, feature informa- tion). We focus primarily on a resource-rich/resource- poor situations, i.e. on language pairs where for one of the languages resources such as a parser are available, but not for the other language. It is outside the scope of this paper to describe our rule learning approach. The interested reader should refer to (Carbonell et al., 2002; Probst, 2002). From the brief description above it should become ap- parent that heavy reliance on feature-tagged dictionaries and/or parsers becomes a new bottleneck for Machine Translation development. Our work focuses on target lan- guages for which there does exist a dictionary, but its en- tries may are initially not tagged with …

Content truncated to 500 words