MI-trigger-based Language Modelling

Language, Information and Computation (PACLIC12), 18-20 Feb, 1998, 195-205 MI-Trigger-based Language Modeling Zhou GuoDong and Lua KimTeng Department of Information Systems and Computer Science National University of Singapore Lower Kent Ridge Road Singapore 119260 {zhouguod, luakt} @iscs.nus.edu.sg ABSTRACT This paper proposes a new MI-Trigger-based modeling approach to capture the preferred relationships between words over a short or long distance. It is implemented by the concept of trigger pair, which is selected by average mutual information and measured by mutual information. Both the distance-independent(DI) and distance-dependent(DD) MI-Trigger-based models are constructed within a window of a size from 1 to 10. It is found that the DD MI-Trigger models have better performance than the DI MI-Trigger models for the same window size and it is better to model the preferred relationships in a distance-dependent way. It is also found that the number of the trigger pairs in an MI-Trigger model can be kept to a reasonable size without losing too much of its modeling power. Finally, it is concluded that the preferred relationships between words are useful to language disambiguation and can be modeled efficiently by the MI-Trigger-based modeling approach. Keyword: MI-Trigger modeling approach, preferred relationship, long-distance context dependency, average mutual information, mutual information. 1 Introduction In natural language there always exist many preferred relationships between words. Lexicographers always use the concepts of collocation, co-occurrence and lexis to describe them. One classic example is "strong" and "powerful". [Halliday66] noticed that these two words were used in different language environments such as "strong tea" and "powerful computer" although they had similar syntax and semantics. Psychologists also have a similar concept: word association. Two highly associated word pairs are "bread/butter" and "doctor/nurse". Psychological experiments in [Meyer+75] indicated that the human' s reaction to a highly associated word pair was stronger and faster than that to a poorly associated word pair. The strength of word association can be measured by mutual information. By computing mutual information between the two words in the word pair, we can get many useful preference information from the corpus, such as the semantic preference between noun and noun(e.g. "doctor/nurse"), the particular preference between adjective and noun(e.g. "strong/tea"), and solid structure(e.g. "the more/the more"). These information are useful for automatic sentence disambiguation. The research in [Hindle+93] showed the role of correlated statistical information in resolving the sentence ambiguity. Consider a simple English sentence: "She (wanted I placed I put) the dress on the rack." For different verbs, the linking directions of the preposition phrase(" on the rack") are different. It can modify the noun(for "wanted") or function as the objective complement of the verbs(for "placed " and "put"). This is the well-known preposition phrase(PP) attachment problem in the analysis of English. Their research showed that a reasonable analysis result can be chosen by comparing the mutual information of the verb- preposition pair("want/on") and • the object-preposition pair("dress/on"). [Magerman+90] used a computational model of mutual information to automatically segment short phrases. [Rosenfeld94] used the concept of trigger pair as the basic information …

Content truncated to 500 words