Word Segmentation as General Chunking

Proceedings of the Fifteenth Conference on Computational Natural Language Learning, pages 39–47, Portland, Oregon, USA, 23–24 June 2011. c⃝2011 Association for Computational Linguistics Word Segmentation as General Chunking Daniel Hewlett and Paul Cohen Department of Computer Science University of Arizona Tucson, AZ 85721 {dhewlett,cohen}@cs.arizona.edu Abstract During language acquisition, children learn to segment speech into phonemes, syllables, morphemes, and words. We examine word segmentation specifically, and explore the possibility that children might have general- purpose chunking mechanisms to perform word segmentation. The Voting Experts (VE) and Bootstrapped Voting Experts (BVE) algo- rithms serve as computational models of this chunking ability. VE finds chunks by search- ing for a particular information-theoretic sig- nature: low internal entropy and high bound- ary entropy. BVE adds to VE the abil- ity to incorporate information about word boundaries previously found by the algorithm into future segmentations. We evaluate the general chunking model on phonemically- encoded corpora of child-directed speech, and show that it is consistent with empirical results in the developmental literature. We argue that it offers a parsimonious alternative to special- purpose linguistic models. 1 Introduction The ability to extract words from fluent speech ap- pears as early as the seventh month in human de- velopment (Jusczyk et al., 1999). Models of this ability have emerged from such diverse fields as lin- guistics, psychology and computer science. Many of these models make unrealistic assumptions about child language learning, or rely on supervision, or are specific to speech or language. Here we present an alternative: a general unsupervised model of chunking that performs very well on word segmen- tation tasks. We will examine the Voting Experts, Bootstrapped Voting Experts, and Phoneme to Mor- pheme algorithms in Section 2. Each searches for a general, information-theoretic signature of chunks. Each can operate in either a fully unsupervised set- ting, where the input is a single continuous se- quence of phonemes, or a semi-supervised setting, where the input is a sequence of sentences. In Sec- tion 4, we evaluate these general chunking methods on phonetically-encoded corpora of child-directed speech, and compare them to a representative set of computational models of early word segmentation. Section 4.4 presents evidence that words optimize the information-theoretic signature of chunks. Sec- tion 5 discusses segmentation methods in light of what is known about the segmentation abilities of children. 2 General Chunking The Voting Experts algorithm (Cohen and Adams, 2001) defines the chunk operationally as a sequence with the property that elements within the sequence predict one another but do not predict elements out- side the sequence. In information-theoretic terms, chunks have low entropy internally and high entropy at their boundaries. Voting Experts (VE) is a lo- cal, greedy algorithm that works by sliding a rel- atively small window along a relatively long input sequence, calculating the internal and boundary en- tropies of sequences within the window. The name Voting Experts refers to the two “ex- perts” that vote on possible boundary locations: One expert votes to place boundaries after se- quences that have low …

Content truncated to 500 words