Unsupervised Dependency Parsing with Acoustic Cues

Transactions of the Association for Computational Linguistics, 1 (2013) 63–74. Action Editor: Brian Roark. Submitted 9/2012; Published 3/2013. c⃝2013 Association for Computational Linguistics. Unsupervised Dependency Parsing with Acoustic Cues John K Pate†‡ j.k.pate@sms.ed.ac.uk Sharon Goldwater† sgwater@inf.ed.ac.uk †ILCC, School of Informatics ‡Department of Computing University of Edinburgh Macquarie University Edinburgh, EH8 9AB, UK Sydney, NSW 2109, Australia Abstract Unsupervised parsing is a difficult task that infants readily perform. Progress has been made on this task using text-based models, but few computational approaches have considered how infants might benefit from acoustic cues. This paper explores the hypothesis that word duration can help with learning syntax. We de- scribe how duration information can be incor- porated into an unsupervised Bayesian depen- dency parser whose only other source of infor- mation is the words themselves (without punc- tuation or parts of speech). Our results, evalu- ated on both adult-directed and child-directed utterances, show that using word duration can improve parse quality relative to words-only baselines. These results support the idea that acoustic cues provide useful evidence about syntactic structure for language-learning in- fants, and motivate the use of word duration cues in NLP tasks with speech. 1 Introduction Unsupervised learning of syntax is difficult for NLP systems, yet infants perform this task routinely. Pre- vious work in NLP has focused on using the implicit syntactic information available in part-of-speech (POS) tags (Klein and Manning, 2004), punctuation (Seginer, 2007; Spitkovsky et al., 2011b; Ponvert et al., 2011), and syntactic similarities between related languages (Cohen and Smith, 2009; Cohen et al., 2011). However, these approaches likely use the data in a very different way from children: neither POS tags nor punctuation are observed during language acquisition (although see Spitkovsky et al. (2011a) and Christodoulopoulos et al. (2012) for encourag- ing results using unsupervised POS tags), and many children learn in a broadly monolingual environment. This paper explores a possible source of information that NLP systems typically ignore: word duration, or the length of time taken to pronounce each word. There are good reasons to think that word dura- tion might be useful for learning syntax. First, the well-established Prosodic Bootstrapping hypothesis (Gleitman and Wanner, 1982) proposes that infants use acoustic-prosodic cues (such as word duration) to help them identify syntactic structure, because prosodic and syntactic structures sometimes coincide. More recently, we proposed (Pate and Goldwater, 2011) that infants might use word duration as a di- rect cue to syntactic structure (i.e., without requir- ing intermediate prosodic structure), because words in high-probability syntactic structures tend to be pronounced more quickly (Gahl and Garnsey, 2004; Gahl et al., 2006; Tily et al., 2009). Like most recent work on unsupervised parsing, we focus on learning syntactic dependencies. Our work is based on Headden et al. (2009)’s Bayesian version of the Dependency Model with Valence (DMV) (Klein and Manning, 2004), using interpo- lated backoff techniques to incorporate multiple infor- mation sources per token. However, whereas Head- den et al. used words and POS tags as input, we use words and …

Content truncated to 500 words