Reconciliation of Unsupervised Clustering, Segmentation and Cohesion

II II II II II II II II II II II Reconciliation of Unsupervised Clustering, Segmentation and Cohesion David M. W. Powers Department of Computer Science The Flinders University of South Australia powers@acm.org Abstract This extended abstract examines the progress of a project on unsupervised language learning, and focuses on two different approaches to segmentation, as well as how cohesion may be generalized from it definitive morpho- syntactic instantiation. It is intended as a discussion paper, and outlines the specific hypotheses currenlty being tested. 1. Introduction and Motivation This extended abstract summarize recent and current work being carried out by my group in relation to unsupervised learning of language. The work described is unsupervised in the sense that • there is no human preproeessing of the raw corpora; • there is no preconceived target grammar; • there is a minimum of imposed formalism; • there is no tuning for particular languages/datasets. The methodology used has been inspired by linguistic, psycholinguistie and cognitivist research in language and vision. Whilst a connectionist framework is not in general used, the approach is intended to be neurologically plausbile, although there is more direct influence from probability theory and information theory, and some experiments have used self-organizing neural nets (Powers, 1989; Schifferdecker, 1994). Our earliest work focussed on learning syntax by finding statistical correlations or using self-organizing time-delay networks (Powers, 1989). Hierarchical grammars were produced by introducing newly found relationships as new candidates for correlation. Reasonable grammars were produced only for training date of consistent short length phrases or sentences, but the experiment lead to significant insights: the closed class elements (function words in the initial experiments) were learned first, and these acted like seeds which expanded into larger and larger grammatically meaningful units. This research was subsequently generalized to a binary clustering approach inspired by Pike's Phonemic (1949) and Tagmemic methodologies (1977). Tokens that function similarly in some sense (phonological, morphological, syntactic or semantic) but represent systematic rather than free variation, will form Complementary Distributions or classes. However, free distributions and distributions based on correlations with information which is unavailable at the current level of analysis will not be distinguishable, and thus complementarity will not be established, and may only be assumed under the hypothesis that there is no such thing as free variation or arbitrariness, and that apparent free variation always has causal roots. The complementary method of Pike is Contrast in Identical or Analogous Environments (CIE/CAE) and assumes that there is extraneous information about whether or not the units belong to the same emic unit or not. If we do not make any use of extra-sentential information, and rather assume that all units with distinct forms at the current level convey distinct information, but where they occur in the same set of contexts (coset) they are treated as similar and belong to the same class and convey some common information. The set of units which earl occur in a contextual coset are Contextual Distributions and have the same basic character as Complementary …

Content truncated to 500 words