Evaluation of Automatic Formant Trackers

Evaluation of Automatic Formant Trackers F. Schiel, Th. Zitzelsberger Bavarian Archive for Speech Signals, Ludwig-Maximilians-Universität Schellingstr. 3, 80799 München, Germany {schiel,tzom}@bas.uni-muenchen.de Abstract Four open source formant trackers, three LPC-based and one based on Deep Learning, were evaluated on the same American English data set VTR-TIMIT. Test data were time-synchronized to avoid differences due to different unvoiced/voiced detection strategies. Default output values of trackers (e.g. producing 500Hz for the first formant, 1500Hz for the second etc.) were filtered from the evaluation data to avoid biased results. Evaluations were performed on the total recording and on three American English vowels [i:], [u] and [ ] ʌ separately. The obtained quality measures showed that all three LPC-based trackers had comparable RSME error results that are about 2 times the inter-labeller error of human labellers. Tracker results were biased considerably (in average too high or low), when the parameter settings of the tracker were not adjusted to the speaker's sex. Deep Learning appeared to outperform LPC-based trackers in general, but not in vowels. Deep Learning has the disadvantage that it requires annotated training material from the same speech domain as the target speech, and a trained Deep Learning tracker is therefore not applicable to other languages. Keywords: formant tracker, evaluation, VTR-TIMIT 1. Introduction This paper presents the methodology and results of a technical evaluation of four open source algorithms for automatic formant tracking (in the following referred to as 'formant trackers'). Currently, a small number of open source formant trackers is widely used by speech scientists and speech engineers. A number of earlier studies evaluated selections of these formant trackers on selected vowels, on words spoken in isolation and/or on speech of a small number of speakers: Derdemezis et al. 2016 evaluated 4 LPC-based formant trackers on 4 different vowels in isolated words uttered by 8 speaker groups; Deng et al, 2006 evaluated their own formant tracker compared to WaveSurfer based on VTR-TIMIT; Harrison, 2004 compared three LPC-based algorithms on data of two speakers. But to our knowledge there exists no objective evaluation of the quality of more than two algorithms based on the same data set comprising a reasonable number of speakers of both genders and fluent speech. Partly this is due to the fact that a manually controlled reference corpus of fluent speech for formant tracks is difficult to obtain. Fortunately, with the publication of the VTR-TIMIT (Vocal Tract Resonances TIMIT) corpus by Deng et al (2006) we are now able to perform such an evaluation, at least for US American English. Aside from the direct comparison of different formant trackers as presented in this study there exist numerous studies that compare the out-come of a single formant tracker to one or more expert annotations, or evaluate the influence of methodological parameters such as the time point of measurement (Kendall & Vaughn 2015), the number of formants or LPC order (Vallabha & Tuller, 2002; Harrison 2004), the signal quality (Rathcke et al. 2016), or even the outcome of different expert groups using …

Content truncated to 500 words