Learning grammatical categories using paradigmatic representations: Substitute words for language acquisition

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 707–716, Osaka, Japan, December 11-17 2016. Learning grammatical categories using paradigmatic representations: Substitute words for language acquisition Mehmet Ali Yatbaz1∗ Volkan Cirik2∗ Aylin K¨untay3 Deniz Yuret3 1Facebook Inc. 1 Hacker Way Menlo Park, CA, USA 2Carnegie Mellon University, Pittsburgh, PA, USA 3Koc¸ University, ˙Istanbul, Turkey {myatbaz,vcirik,akuntay,dyuret}@ku.edu.tr Abstract Learning word categories is a fundamental task in language acquisition. Previous studies show that co-occurrence patterns of preceding and following words are essential to group words into categories. However, the neighboring words, or frames, are rarely repeated exactly in the data. This creates data sparsity and hampers learning for frame based models. In this work, we propose a paradigmatic representation of word context which uses probable substitutes instead of frames. Our experiments on child-directed speech show that models based on probable substitutes learn more accurate categories with fewer examples compared to models based on frames. 1 Introduction Children abstract grammatical rules from individual words (e.g. baby, talk) and eventually apply them to word categories (e.g. noun, verb, adverb). A word category represents a group of words that can be substituted for one another without altering the grammatical appropriateness of a sentence. Learning word categories is an important step in language development. The Distributional Hypothesis (Harris, 1954) suggests that words occurring in similar contexts tend to have similar meanings and grammatical properties. Studies on extraction of word categories have shown that distributional information of word co-occurrences is a reliable cue for the learning of word categories (Mintz, 2003; St Clair et al., 2010; Redington et al., 1998). Children need to extract word categories from incoming speech stream in order to be able to predict how words behave in various grammatical contexts and to produce words in appropriate grammatical constructions. Children tend to form word categories that group words used in similar contexts. To judge how similar two contexts are, one can use syntagmatic or paradigmatic representations of the word context: A syntagmatic representation is based on the neighbors of the target word whereas a paradigmatic representation uses potential substitutes for the target word. In this paper, we hypothesize that children judge context similarity using a paradigmatic represen- tation: a context is similar to another if the same words can be substituted in both. The following two examples1 illustrate the advantage of paradigmatic representations in uncovering latent similarities where a syntagmatic representation would fail to see any overt similarities. The word “you” from the first sen- tence and the word “I” from the second sentence have no common neighbors within the same sentence. The paradigmatic representation, shown below the sentences as substitute word probabilities, captures the similarity of these contexts by suggesting similar top substitutes for both: (1) “they fall out when you put it in the box .” you: you(.8188), I(.1027), they(.0408), we(.0146) . . . (2) “what have I got here ?” I: we(.8074), you(.1213), I(.0638), they(.0073) . . . This work is licenced under a Creative Commons Attribution …

Content truncated to 500 words