Distributed Representation of Chinese Collocation

Distributed Representation of Chinese Collocation Bo Xia Endong Xun Beijing Language and Culture University Beijing Language and Culture University blcuxiabo@126.com edxun@blcu.edu.cn Abstract We provide computational linguistics an unsupervised algorithm for Collocation Representing in vector space using segmented corpora. By collocation, we mean a directed association of head word and dependency word that constructed by fully random combine in a sentence. Our algorithm represents dependency words by dense vectors that are trained to predict contexts of the head word. We show that these vectors provide high performance for extracting collocation similarities in syntactic and semantic. The quality of work results performs effective on our test set, which is measured in a verb-verb phrase collocation prediction task. 1 Introduction “You shall know a word by the company it keeps” (Firth, 1957). As an important role of language phenomenon, lexical collocations is still far from work in nature language processing such as parsing, word sense disambiguation, and topic modeling (Carlos Ramisch, 2013). For example, a parser that lacks sufficient knowledge of verb-particle constructions might correctly assign look up the tower two interpretations (“glance up at the tower” vs. “consult a reference book about the tower”), but fail to treat the subtly different look the tower up as unambiguous (“consult a reference book” interpretation only) (Ivan A. Sag, 2002). Composed of a head word and a dependency one, collocations are restricted lexical co- occurrences of two syntactically bound lexical elements (Kilgarriff, 2006).A large body of work, known as Multiword expressions (MWEs) made up of at least 2 words, has studied the expand phenomena of collocation which are sequence of words that acts as a single unit at some level of linguistic analysis (Calzolari et al., 2002). Despite popularity of collocations and various labels, it is a limitation of collocation representation that many of collocations present an unusual structure (Iñaki Alegria, 2004), especially for languages with large vocabularies and many rare words. On the other hand, disambiguation of collocation in context is frequent in corpora (e.g., bus stop, as in Does the bus stop here? vs. The bus stop is here.) (Miriam R. L. Petruck, 2015). Since both of them are hard for structured collocation representation, the representation learning of collocations become our inspiration in the recent work of machine learning community. In this paper, we study in depth one collocation phenomena: verb-verb phrase (In English, it can be compared to phenomena of infinitives and gerund as verb implement). We focus on distributed representations of collocations learned by neural networks, as it trained on corpora and huge amounts of Verb-verb phrase recognition with sparsity. We develop new model architectures that preserve the collocation regularities given contexts. We present collocations representation similarity goes beyond simple syntactic regularities. For measuring both syntactic and semantic regularities, we present experiments of predicting Verb-verb phrase recognition in test set that labeled by students of Applied Linguistics, and show that linguistic regularities can be learned with considerable accuracy. Moreover, we discuss how training time and accuracy depends on the amount of …

Content truncated to 500 words