Minimally Supervised Number Normalization

Minimally Supervised Number Normalization Kyle Gorman and Richard Sproat Google, Inc. 111 8th Ave., New York, NY, USA Abstract We propose two models for verbalizing num- bers, a key component in speech recognition and synthesis systems. The first model uses an end-to-end recurrent neural network. The sec- ond model, drawing inspiration from the lin- guistics literature, uses finite-state transducers constructed with a minimal amount of training data. While both models achieve near-perfect performance, the latter model can be trained using several orders of magnitude less data than the former, making it particularly useful for low-resource languages. 1 Introduction Many speech and language applications require text tokens to be converted from one form to another. For example, in text-to-speech synthesis, one must con- vert digit sequences (32) into number names (thirty- two), and appropriately verbalize date and time ex- pressions (12:47 → twelve forty-seven) and abbre- viations (kg → kilograms) while handling allomor- phy and morphological concord (e.g., Sproat, 1996). Quite a bit of recent work on SMS (e.g., Beaufort et al., 2010) and text from social media sites (e.g., Yang and Eisenstein, 2013) has focused on detect- ing and expanding novel abbreviations (e.g., cn u plz hlp). Collectively, such conversions all fall under the rubric of text normalization (Sproat et al., 2001), but this term means radically different things in differ- ent applications. For instance, it is not necessary to detect and verbalize dates and times when preparing social media text for downstream information extrac- tion, but this is essential for speech applications. While expanding novel abbreviations is also im- portant for speech (Roark and Sproat, 2014), num- bers, times, dates, measure phrases and the like are far more common in a wide variety of text genres. Following Taylor (2009), we refer to cate- gories such as cardinal numbers, times, and dates— each of which is semantically well-circumscribed— as semiotic classes. Some previous work on text nor- malization proposes minimally-supervised machine learning techniques for normalizing specific semi- otic classes, such as abbreviations (e.g., Chang et al., 2002; Pennell and Liu, 2011; Roark and Sproat, 2014). This paper continues this tradition by con- tributing minimally-supervised models for normal- ization of cardinal number expressions (e.g., ninety- seven). Previous work on this semiotic class include formal linguistic studies by Corstius (1968) and Hur- ford (1975) and computational models proposed by Sproat (1996; 2010) and Kanis et al. (2005). Of all semiotic classes, numbers are by far the most im- portant for speech, as cardinal (and ordinal) num- bers are not only semiotic classes in their own right, but knowing how to verbalize numbers is important for most of the other classes: one cannot verbalize times, dates, measures, or currency expressions with- out knowing how to verbalize that language’s num- bers as well. One computational approach to number name ver- balization (Sproat, 1996; Kanis et al., 2005) employs a cascade of two finite-state transducers (FSTs). The first FST factors the integer, expressed as a digit se- quence, into sums of products of powers of …

Content truncated to 500 words