Markov random field based English Part-Of-Speech tagging system

Marker random field based English Part-Of-Speech tagging system Sung-Young Jung , Young C. Park, Key-Sun Choi and Youngwhan Kim* Comput;er Science ])el)a.rtment Korea Advanced [nstitul, e of Science and q%chnology Taejon, Korea *Multimedia Re.search Ibaboratories Korea ~[>lecom { chopin,ycpark,kschoi}(ci!csone.kaist.ac.kr Abstract Probabilistic models have been widely used for natural language processing. Part-of-speech tagging, which assigns the most likely tag to each word in a given sentence, is one. of tire problems which can be solved by statisticM ap- proach. Many researchers haw~ tried to solve the problem by hidden Marker model (HMM), which is well known as one of the statistical models. But it has many difficulties: integrating hetero- geneous information, coping with data sparseness prohlem, and adapting to new environments. In this paper, we pro- pose a Markov radom field (MRF) model based approach to the tagging problem. The MRF provides the base frame to combine various statistical information with maximum entropy (ME) method. As Gibbs distribution can be used to describe a posteriori probability of tag- ging, we use it in ma.ximum a posteri- ori (MAP) estimation of optimizing pro- cess. Besides, several tagging models are developed to show the effect of adding information. Experimental results show that the performance of the tagger gets improved as we add more statistical in- formation, and that Mt{F-based tagging model is better than ttMM based tagging model in data sparseness problem. 1 Introduction Part-of-speech tagging is to assign the correct tag to each word in the context of the sentence. '['here are three main approaches in tagging problem: rule-based approach (Klein and Simmons 1%3; Brodda 1982; Paulussen and Martin 1992; Brill et al. 1990), statistical approach (Church :1988; Merialdo 1994; Foster 1991; Weischedel et al. 1993; Kupiec 1992) and connectionist approach (Benello et al. 1989; Nakanmra et al. 1989). In these approaches, statistical approach has the fol- lowing advantages : • a theoretical framework is provided • automatic learning facility is provided • the probabilities provide a straightforward way to disambiguate Many information sources must be combined to solve tagging problem with statistical approach. It is a significant assumption that tire correct tag can generally be chosen from I.he local context. Not only local sequences of words and tags are needed to solve tagging problem, but syntax, se- mantic, and morphological level information is also required in general. Usually information sources such as t)igram, trigram and migra.m are used in the tagging systems which are based on statistical method. Traditionally, linear interpolation an(t its variants have been used to combine the informa- tion sources, })tit these are shown to be seriously deficient. ME (Maximum Entropy) estimation method provides the facility to combine several informa- tion sources. Each inR)rmation source gives rise to a set of constraints, to be imposed on the con> bined estimate. The function with the highest en- tropy within the constraints is the ME solution. Given consistent statistical evidence, a unique M E solution is guaranteed to exist and an iteratiw~" al- gorithm is provided. MRF (Marker random field) …

Content truncated to 500 words