Evaluating Low-Level Speech Features Against Human Perceptual Data

Evaluating Low-Level Speech Features Against Human Perceptual Data Caitlin Richter Dept. of Linguistics University of Pennsylvania ricca@sas.upenn.edu Naomi H. Feldman Dept. of Linguistics and UMIACS University of Maryland nhf@umd.edu Harini Salgado Dept. of Computer Science Pomona College harini.salgado@gmail.com Aren Jansen HLTCOE Johns Hopkins University arenjansen1@gmail.com Abstract We introduce a method for measuring the corre- spondence between low-level speech features and human perception, using a cognitive model of speech perception implemented directly on speech recordings. We evaluate two speaker normalization techniques using this method and find that in both cases, speech features that are normalized across speakers predict hu- man data better than unnormalized speech fea- tures, consistent with previous research. Re- sults further reveal differences across normal- ization methods in how well each predicts hu- man data. This work provides a new frame- work for evaluating low-level representations of speech on their match to human perception, and lays the groundwork for creating more eco- logically valid models of speech perception. Index Terms: speech perception, automatic speech recognition, speaker normalization, cognitive models 1 Introduction Understanding the features that listeners extract from the speech signal is a critical part of understand- ing phonetic learning and perception. Different fea- ture spaces imply different statistical distributions of speech sounds in listeners’ input (Figure 1), and these statistical distributions of speech sounds influ- ence speech perception in both adults and infants (Clayards et al., 2008; Maye et al., 2002). The performance of automatic speech recognition (ASR) systems is also affected by the way in which these systems represent the speech signal. For ex- ample, changing the signal processing methods that are used to extract features from the speech wave- form (Hermansky, 1990; Hermansky and Morgan, 1994) and applying speaker normalization techniques to these features (Wegmann et al., 1996; Povey and Saon, 2006) can improve the performance of a recog- nizer. Recently there has been considerable interest in representation learning, in which new features that are created through exposure to data from the lan- guage to be recognized, or through exposure to other languages, lead to better system performance (Gr´ezl et al., 2014; Heigold et al., 2013; Kamper et al., 2015; Sercu et al., 2016; Thomas et al., 2012; Wang et al., 2015). It is potentially useful to know how closely the feature representations in ASR resemble those of human listeners, particularly for low-resource set- tings, where systems rely heavily on these features to guide generalization across speakers and dialects. In this paper we introduce a method for measuring the correspondence between low-level speech fea- ture representations and human speech perception. This allows us to compare different feature spaces in terms of how well the locations of sounds in the space can predict listeners’ perception of acoustic similarity. Our method has potential relevance both in ASR, for understanding how well the feature rep- resentations in ASR systems capture the similarity structure that guides human perception, and in cogni- tive science, for evaluating hypotheses regarding the feature spaces that human listeners use. We evaluate the ability of …

Content truncated to 500 words