Assessing the Prosody of Non-Native Speakers of English: Measures and Feature Sets

Assessing the Prosody of Non-Native Speakers of English: Measures and Feature Sets Eduardo Coutinho 1,2, Florian H¨onig 3, Yue Zhang 1, Simone Hantke 4 Anton Batliner 3,5, Elmar N¨oth 3, Bj¨orn Schuller 1,5 1Department of Computing, Imperial College London, UK 2Department of Music, University of Liverpool, UK 3Pattern Recognition Lab, Friedrich-Alexander University Erlangen-Nuremberg, Germany 4Machine Intelligence & Signal Processing Group, TUM, Munich, Germany 5Chair of Complex & Intelligent Systems, University of Passau, Germany e.coutinho@imperial.ac.uk Abstract In this paper, we describe a new database with audio recordings of non-native (L2) speakers of English, and the perceptual evaluation experiment conducted with native English speakers for assessing the prosody of each recording. These annotations are then used to compute the gold standard using different methods, and a series of regression experiments is conducted to evaluate their impact on the performance of a regression model predicting the degree of naturalness of L2 speech. Further, we compare the relevance of different feature groups modelling prosody in general (without speech tempo), speech rate and pauses modelling speech tempo (fluency), voice quality, and a variety of spectral features. We also discuss the impact of various fusion strategies on performance. Overall, our results demonstrate that the prosody of non-native speakers of English as L2 can be reliably assessed using supra-segmental audio features; prosodic features seem to be the most important ones. Keywords: Index Terms: non-native speech, prosody, feature evaluation 1. Introduction Non-native (L2) speakers of English diverge from native English speakers in terms of linguistic (e.g, morphology, syntax, lexicon) and phonetic aspects. Phonetic aspects comprise segmental and supra-segmental (prosodic) phe- nomena. There are two main tasks within automatic speech processing where an assessment of non-native traits can help. First, the performance of Automatic Speech Recog- nition (ASR) degrades drastically while processing non- native speech. A central aspect here is the mismatch be- tween training data from native speech and test data from non-native speech (Van Compernolle, 2001), preventing the acoustic models to capture the variability and ‘innovations’ introduced by L2 speakers. By assessing the degree of na- tiveness in general, and by specifying the speaker’s L1, it is possible to build speaker- and/or speaker-group spe- cific acoustic models. Second, such an assessment can be employed within Computer-Aided Pronunciation Train- ing (CAPT) for the automatic screening and assessment of learners. In this paper, we describe a new database with audio record- ings of L2 English speakers, and the perceptual evalua- tion experiment conducted with native English speakers to obtain judgements of the naturalness of the pronunciation of each recording with respect to supra-segmental char- acteristics. These annotations are then used to compute the gold standard using different methods, and a series of regression experiments is conducted to evaluate their im- pact on the performance of a regression model predicting the degree of naturalness of L2 speech. We use the pub- licly available general-purpose feature set openSMILE (Ey- ben et al., 2013) to compute a large set of acoustic fea- tures. Additionally, to add some more specialised (but still text-independent …

Content truncated to 500 words