Linguistic Knowledge Extraction from Real Language Behavior

Linguistic Knowledge Extraction from Real Language Behavior R.Shirai and T.Ilamada (Department of Electrical Engineering, Waseda University) (3-4-10hkubo ShinJuku-ku Tokyo Japan) Abstract -- An approach to extract linguis- tic knowledge from real language behavior is described. This method depends on the extraction of word relations, patterns of which are obtained by structuring the dependency relations in sentences called Kakari-Uke relation in Japanese. As the first step of this approach, an experiment of a word classification utilizing those patterns was made on tile 4178 sentences of real language data. A system was made to analyze dependency structure of sentences uti 1 izing the knowledge base obtained through this word classification and the effectiveness of the knowledge base ~4as evaluated. To develop this approach further, the relation matrix which captures multiple interaction of words is proposed. 1. Introduction In natural language processing, one of the major problems to be solved is how to describe linguistic and semantic knowledge in the sye]tem. If we nse no particular technique and capture the behavior in real I anguage as i t i s, the number of FU]es. concepts and relations to be arranged may expand so much° But those things contain all essential and primitive elements of language that we want to find out at ]east. In this Paper, it is considered to extract primitive elements from real linguistic behavior, and apply the elements to analysis sentence. As the above-mentioned elements, we use a relation between words. (lt is called Kakari-Uke relation in Japanese.) SUBJ John opened the door vith this ke~, t NST Fig.l 9ependency Relation Structure (Kakari-Uke Relations) 2. Clustering of Words 2.1. Clustering Method The process of the word classification based on the pattern of relations is done as follows. First, numbers of sentences are provided and Kakari-Uke relations are given to them. We call those sentences text data. Next we get the source side and the sink side pattern of relations for each word appearing in the text data. Then we calculate a distance between words. The d i s lance i s defi ned as a correspondence between the patterns themselves and the frequency of each relation making the patterns. Words are classified by a clustering algorithm using this distance. The distance has two types; one for the source side patterns and the other for' the sink side patterns. For each word, two clustering processes are applied correspond- ing to those two types ot distances. In this paper, the dependency strt.lcture is called as the knowledge base. 2.2. Re.sul {s We made an experi ment of word clustering on the 4178 sentences of text data quoted from computer manuals. In this experiment, a special treatment was taken for compound words to ensure information. There are many compound words in Japanese sentences which are made by combining words and act as one word. They are called Fuku9o- go in Japanese. If we great them all as different from each other, many words appear rarely, so that the relating pat.terns of each word …

Content truncated to 500 words