Retrieving Collocations by Co-Occurrences and Word Order Constraints

Retrieving Collocations by Co-occurrences and Word Order Constraints Sayori Shimohata, Toshiyuki Sugio and Junji Nagata Kansai Laboratory, Research &: Development Group Oki Electric Industry Co., Ltd. Crystal Tower 1-2-27, Shiromi, Chuo-ku, Osaka, 540, Japan { sayori, sugio, nagat a} ©kansai. oki. co. j p Abstract In this paper, we describe a method for automatically retrieving collocations from large text corpora. This method retrieve collocations in the following stages: 1) ex- tracting strings of characters as units of collocations 2) extracting recurrent combi- nations of strings in accordance with their word order in a corpus as collocations. Through the method, various range of col- locations, especially domain specific collo- cations, are retrieved. The method is prac- tical because it uses plain texts without any information dependent on a language such as lexical knowledge and parts of speech. 1 Introduction A collocation is a recurrent combination of words, ranging from word level to sentence level. In this pa- per, we classify collocations into two types according to their structures. One is an uninterrupted colloca- tion which consists of a sequence of words, the other is an interrupted collocation which consists of words containing one or several gaps filled in by substi- tutable words or phrases which belong to the same category. The features of collocations are defined as follows: • collocations are recurrent • collocations consist of one or several lexical units • order of units are rigid in a collocation. For language processing such as machine trans- lation, a knowledge of domain specific collocations is indispensable because what collocations mean are different from their literal meaning and the usage and meaning of a collocation is totally dependent on each domain. In addition, new collocations are produced one after another and most of them are technical jargons. There has been a growing interest in corpus-based approaches which retrieve collocations from large corpora (Nagao and Mori, 1994), (Ikehara et al., 1996) (Kupiec, 1993), (Fung, 1995), (Kitamura and Matsumoto, 1996), (Smadja, 1993), (Smadja et al., 1996), (Haruno et al., 1996). Although these ap- proaches achieved good results for the task consid- ered, most of them aim to extract fixed collocations, mainly noun phrases, and require the information which is dependent on each language such as dictio- naries and parts of speech. From a practical point of view, however, a more robust and flexible approach is desirable. We propose a method to retrieve interrupted and uninterrupted collocations by the frequencies of co-occurrences and word order constraints from a monolingual corpus. The method comprises two stages: the first stage extracts sequences of words (or characters) t from a corpus as units of colloca- tions and the second stage extracts recurrent com- binations of units and constructs collocations by ar- ranging them in accordance with word order in the corpus. 2 Algorithm 2.1 Extracting units of collocation (Nagao and Mori, 1994) developed a method to calculate the frequencies of strings composed of n characters(a grams). Since this method generates all n-character strings appeared in a text, the …

Content truncated to 500 words