Improving homograph disambiguation with supervised machine learning

Improving homograph disambiguation with supervised machine learning Kyle Gorman, Gleb Mazovetskiy, Vitaly Nikolaev Google Inc. {kbg,glebm,vitalyn}@google.com Abstract We describe a pre-existing rule-based homograph disambiguation system used for text-to-speech synthesis at Google, and compare it to a novel system which performs disambiguation using classifiers trained on a small amount of labeled data. An evaluation of these systems, using a new, freely available English data set, finds that hybrid systems (making use of both rules and machine learning) are significantly more accurate than either hand-written rules or machine learning alone. The evaluation also finds minimal performance degradation when the hybrid system is configured to run on limited-resource mobile devices rather than on production servers. The two best systems described here are used for homograph disambiguation on all US English text-to-speech traffic at Google. Keywords: Homograph disambiguation, machine learning, text normalization, text-to-speech synthesis 1. Introduction Despite substantial progress in applying machine learn- ing to text normalization and linguistic analysis for text- to-speech synthesis (TTS), it is still the case that most front-end processing in real-world TTS systems is done by language-specific hand-written rules. While rule systems provide an high degree of interpretability, they may require a great deal of development effort to obtain reasonable accu- racy. Therefore, rule-based components represent substan- tial barriers for both quality control and internationalization. 1.1. Homograph disambiguation When text input is sent to the Google TTS engine, it is first tokenized, and then pronunciations are selected for these to- kens (Ebden and Sproat, 2014). For most tokens—such as in-vocabulary words with a single pronunciation—this re- quires only dictionary lookup. But other types of tokens, so-called semiotic classes (Taylor, 2009)—such as numbers, currencies and measures, dates and times, etc.—and out-of- vocabulary words, require additional language-specific pro- cessing (Sproat et al., 1992; Sproat et al., 2001). One par- ticularly challenging class of tokens are homographs, pol- ysemous words pronounced differently depending on the intended sense. One must analyze the context in which a homograph occurs to select a contextually appropriate pro- nunciation. 1.2. Sources of homography Homography occurs any time two words pronounced dis- tinctly are spelled the same. This may arise due to inflec- tional processes not indicated orthographically, as in the En- glish irregular verb read, pronounced as either the past [ɹɛd] or the present [ɹiːd]. Alternatively, homographs may repre- sent phonologically and semantically distinct lexical items which just happen to share a spelling, as in bow, pronounced either as [boʊ] or [baʊ]. We refer to these categories of homography as morphosyntactic and lexical, respectively.1 1 This cuts across the traditional distinction between those ho- mographs which can or cannot be disambiguated by part of speech (Yarowsky, 1997), since the distinct pronunciations of the latter category may or may not have overlapping parts of speech. For in- stance, the lexical homograph console is largely disambiguated by part of speech, but both pronunciations of the lexical homograph bass—[beɪs] and [bæs]—are usually nouns. One may also discern a mixed category for those homo- graphs which carry both morphosyntactic and semantic …

Content truncated to 500 words