Incorporation of WordNet Features to n-gram Features in a Language Modeler

Incorporation of WordNet Features to n-gram Features in a Language Modeler * Kathleen L. Go a and Solomon L. See a a De La Salle University – Manila 2401 Taft Avenue, Malate 1004 Manila, Philippines go.kathleen@gmail.com, sees@dlsu.edu.ph Abstract. n-gram language modeling is a popular technique used to improve performance of various NLP applications. However, it still faces the “curse of dimensionality” issue wherein word sequences on which the model will be tested are likely to be different from those seen during training (Bengio et al., 2003). An approach that incorporates WordNet to a trigram language modeler has been developed to address this issue. WordNet was used to generate proxy trigrams that may be used to reinforce the fluency of the given trigrams. Evaluation results reported a significant decrease in model perplexity showing that the new method, evaluated using the English language in the business news domain, is capable of addressing the issue. The modeler was also used as a tool to rank parallel translations produced by multiple Machine Translation systems. Results showed a 6-7% improvement over the base approach (Callison-Burch and Flournoy, 2001) in correctly ranking parallel translations. Keywords: language modeling, statistical methods, translation quality, WordNet 1. Introduction n-gram language modeling is a popular statistical language modeling (SLM) technique used to improve performance of various natural language processing (NLP) applications such as speech recognition (SR), machine translation (MT), and information retrieval (IR). However, it still faces the “curse of dimensionality” issue wherein word sequences (i.e. n-grams) on which the model will be tested are likely to be different from those seen during training (Bengio et al., 2003). This means that these sequences would always be assigned low probabilities. There have already been attempts to address this problem. A number of works made use of smoothing techniques such as the one presented in (Callison-Burch and Flournoy, 2001) which made use of linear interpolation of trigram, bigram and unigram probabilities. (Bengio et al., 2003) presents an approach that combines neural networks and smoothing techniques to a trigram model while the work in (Brockett et al., 2001) made use of additional linguistic features, such as syntax trees, in order to construct a decision tree that will classify sentences. Another possible solution is to make use of other linguistic resources such as WordNet (Fellbaum et al., 2006) and devise an approach that combines WordNet features (i.e. synsets and their relations) with the n-grams used in language modelers. By incorporating WordNet features, the language modeler can generate related sequences which can possibly give it a higher score even if there are no exact matches seen during training. (Hoberman and Rosenfeld, 2002) presents a study on the integration of WordNet features to address the data sparseness of nouns in bigrams. The study covered the IS-A relationship of nouns and reported * Copyright 2008 by Kathleen L. Go and Solomon L. See 179 22nd Pacific Asia Conference on Language, Information and Computation, pages 179–188 improvement in the language model perplexity, although the improvement was below expectation. This …

Content truncated to 500 words