Combining Manual and Automatic Prosodic Annotation for Expressive Speech Synthesis

Combining Manual and Automatic Prosodic Annotation for Expressive Speech Synthesis Sandrine Brognaux1,2,3,4, Thomas Franc¸ois1,4, Marco Saerens2 1Cental, IL&C, Universit´e catholique de Louvain (UCL), Belgium 2ICTEAM, Universit´e catholique de Louvain (UCL), Belgium 3 TCTS, Universit´e de Mons (UMons) 4 FNRS, Belgium sandrine.brognaux@umons.ac.be, thomas.francois@uclouvain.be, marco.saerens@uclouvain.be Abstract Text-to-speech has long been centered on the production of an intelligible message of good quality. More recently, interest has shifted to the generation of more natural and expressive speech. A major issue of existing approaches is that they usually rely on a manual annotation in expressive styles, which tends to be rather subjective. A typical related issue is that the annotation is strongly influenced – and possibly biased – by the semantic content of the text (e.g. a shot or a fault may incite the annotator to tag that sequence as expressing a high degree of excitation, independently of its acoustic realization). This paper investigates the assumption that human annotation of basketball commentaries in excitation levels can be automatically improved on the basis of acoustic features. It presents two techniques for label correction exploiting a Gaussian mixture and a proportional-odds logistic regression. The automatically re-annotated corpus is then used to train HMM-based expressive speech synthesizers, the performance of which is assessed through subjective evaluations. The results indicate that the automatic correction of the annotation with Gaussian mixture helps to synthesize more contrasted excitation levels, while preserving naturalness. Keywords: speech synthesis, expressive style annotation, machine learning 1. Introduction Text-to-speech has long been centered on the produc- tion of an intelligible message of good quality. How- ever, as early as the 90’s, some early work already investigated the issue of generating more natural and expressive speech (Murray and Arnott, 1993). Today, this has become a major goal in the field as it comes as an answer to the widespread criticism towards cur- rent speech synthesizers, which have been said to be unnatural and to lack fundamental human components (Kim et al., 2004; Campbell, 2006). In this framework, a large body of research has fo- cused on the synthesis of basic emotions like happi- ness or anger (Yamagishi et al., 2004; Hirose et al., 2005; Qin et al., 2006), which can be seen as a sim- plistic reduction of the expressive naturalness lacking in speech synthesizers (Campbell, 2006). Conversely, we believe that synthesizers should include other types of expressiveness, such as various speaking styles (e.g. TV speech, sports commentaries, political discourse) or attitudes. Surprinsingly, this topic has raised much less inter- est. Lorenzo-Trueba et al. (2013) and Obin et al. (2011) targeted HMM-based speech synthesis of var- ious speaking styles like broadcast news or political speech. In the same vein, Eyben et al. (2012) pro- posed modeling different speaking styles found in au- diobooks. The synthesis of sports commentaries has also aroused some interest (Krstulovi´c et al., 2007; Picart et al., 2013; Brognaux et al., 2013). Specif- ically, it is proposed in Picart et al. (2013), to anno- tate basketball commentaries in various excitation lev- els like “Negative tension”, …

Content truncated to 500 words