Learning to Say It Well: Reranking Realizations by Predicted Synthesis Quality

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 1113–1120, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Learning to Say It Well: Reranking Realizations by Predicted Synthesis Quality Crystal Nakatsu and Michael White Department of Linguistics The Ohio State University Columbus, OH 43210 USA fcnakatsu,mwhiteg@ling.ohio-state.edu Abstract This paper presents a method for adapting a language generator to the strengths and weaknesses of a synthetic voice, thereby improving the naturalness of synthetic speech in a spoken language dialogue sys- tem. The method trains a discriminative reranker to select paraphrases that are pre- dicted to sound natural when synthesized. The ranker is trained on realizer and syn- thesizer features in supervised fashion, us- ing human judgements of synthetic voice quality on a sample of the paraphrases rep- resentative of the generator’s capability. Results from a cross-validation study indi- cate that discriminative paraphrase rerank- ing can achieve substantial improvements in naturalness on average, ameliorating the problem of highly variable synthesis qual- ity typically encountered with today’s unit selection synthesizers. 1 Introduction Unit selection synthesis1—a technique which con- catenates segments of natural speech selected from a database—has been found to be capable of pro- ducing high quality synthetic speech, especially for utterances that are similar to the speech in the database in terms of style, delivery, and coverage (Black and Lenzo, 2001). In particular, in the lim- ited domain of a spoken language dialogue sys- tem, it is possible to achieve highly natural synthe- sis with a purpose-built voice (Black and Lenzo, 2000). However, it can be difficult to develop 1See e.g. (Hunt and Black, 1996; Black and Taylor, 1997; Beutnagel et al., 1999). a synthetic voice for a dialogue system that pro- duces natural speech completely reliably, and thus in practice output quality can be quite variable. Two important factors in this regard are the label- ing process for the speech database and the direc- tion of the dialogue system’s further development, after the voice has been built: when labels are as- signed fully automatically to the recorded speech, label boundaries may be inaccurate, leading to un- natural sounding joins in speech output; and when further system development leads to the genera- tion of utterances that are less like those in the recording script, such utterances must be synthe- sized using smaller units with more joins between them, which can lead to a considerable dropoff in quality. As suggested by Bulyko and Ostendorf (2002), one avenue for improving synthesis quality in a di- alogue system is to have the system choose what to say in part by taking into account what is likely to sound natural when synthesized. The idea is to take advantage of the generator’s periphrastic ability:2 given a set of generated paraphrases that suitably express the desired content in the dialogue context, the system can select the specific para- phrase to use as its response according to the pre- dicted quality of the speech synthesized for that paraphrase. In this way, …

Content truncated to 500 words