Modelling function words improves unsupervised word segmentation

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 282–292, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Modelling function words improves unsupervised word segmentation Mark Johnson1,2, Anne Christophe3,4, Katherine Demuth2,6 and Emmanuel Dupoux3,5 1 Department of Computing, Macquarie University, Sydney, Australia 2 Santa Fe Institute, Santa Fe, New Mexico, USA 3 Ecole Normale Sup´erieure, Paris, France 4 Centre National de la Recherche Scientifique, Paris, France 5 Ecole des Hautes Etudes en Sciences Sociales, Paris, France 6 Department of Linguistics, Macquarie University, Sydney, Australia Abstract Inspired by experimental psychological findings suggesting that function words play a special role in word learning, we make a simple modification to an Adaptor Grammar based Bayesian word segmenta- tion model to allow it to learn sequences of monosyllabic “function words” at the beginnings and endings of collocations of (possibly multi-syllabic) words. This modification improves unsupervised word segmentation on the standard Bernstein- Ratner (1987) corpus of child-directed En- glish by more than 4% token f-score com- pared to a model identical except that it does not special-case “function words”, setting a new state-of-the-art of 92.4% to- ken f-score. Our function word model as- sumes that function words appear at the left periphery, and while this is true of languages such as English, it is not true universally. We show that a learner can use Bayesian model selection to determine the location of function words in their lan- guage, even though the input to the model only consists of unsegmented sequences of phones. Thus our computational models support the hypothesis that function words play a special role in word learning. 1 Introduction Over the past two decades psychologists have in- vestigated the role that function words might play in human language acquisition. Their experiments suggest that function words play a special role in the acquisition process: children learn function words before they learn the vast bulk of the asso- ciated content words, and they use function words to help identify context words. The goal of this paper is to determine whether computational models of human language acqui- sition can provide support for the hypothesis that function words are treated specially in human language acquisition. We do this by comparing two computational models of word segmentation which differ solely in the way that they model function words. Following Elman et al. (1996) and Brent (1999) our word segmentation models identify word boundaries from unsegmented se- quences of phonemes corresponding to utterances, effectively performing unsupervised learning of a lexicon. For example, given input consisting of unsegmented utterances such as the following: j u w ɑ n t t u s i ð ə b ʊ k a word segmentation model should segment this as ju wɑnt tu si ðə bʊk, which is the IPA representation of “you want to see the book”. We show that a model equipped with the abil- ity to learn some rudimentary properties of the target language’s function words is able to learn the vocabulary of that …

Content truncated to 500 words