On the Use of Tied-Mixture Distributions

ON THE USE OF TIED-MIXTURE DISTRIBUTIONS Owen Kimball, Mari Ostendorf Electrical, Computer and Systems Engineering Boston University, Boston, MA 02215 ABSTRACT Tied-mixture (or semi-continuous) distributions are an im- portant tool for acoustic modeling, used in many high- performance speech recognition systems today. This paper provides a survey of the work in this area, outlining the different options available for tied mixture modeling, intro- ducing algorithms for reducing training time, and provid- ing experimental results assessing the trade-offs for speaker- independent recognition on the Resource Management task. Additionally, we describe an extension of tied mixtures to segment-level distributions. 1. INTRODUCTION Tied-mixture (or semi-continuous) distributions have rapidly become an important tool for acoustic model- ing in speech recognition since their introduction by Huang and Jack [1] and nellegarda and iahamoo [2], finding widespread use in a number of high-performance recognition systems. Tied mixtures have a number of advantageous properties that have contributed to their success. Like discrete, "non-parametric" distributions, tied mixtures can model a wide range of distributions including those with an "irregular shape," while retain- ing the smoothed form characteristic of simpler para- metric models. Additionally, because the component distributions of the mixtures are shared, the number of free parameters is reduced, and tied-mixtures have been found to produce robust estimates with relatively small amounts of training data. Under the general heading of tied mixtures, there are a number of possible choices of parameterization that lead to systems with different characteristics. This paper outlines these choices and provides a set of controlled experiments assessing trade- otis in speaker-independent recognition on the Resource Management corpus in the context of the stochastic seg- ment model (SSM). In addition, we introduce new vari- ations on training algorithms that reduce computational requirements and generalize the tied mixture formalism to include segment-level mixtures. 2. PREVIOUS WORK A central problem in the statistical approach to speech recognition is finding a good model for the probabil- ity of acoustic observations conditioned on the state in hidden-Markov models (HMM), or for the case of the SSM, conditioned on a region of the model. Some of the options that have been investigated include discrete dis- tributions based on vector quantization, as well as Gaus- sian, Gaussian mixture and tied-Gaussian mixture dis- tributions. In tied-mixture modeling, distributions are modeled as a mixture of continuous densities, but unlike ordinary, non-tied mixtures, rather than estimating the component Gaussian densities separately, each mixture is constrained to share the same component densities with only the weights differing. The probability density of observation vector x conditioned on being in state i is thus p(x Is = i) = Z wikpk(x). (1) k Note that the component Gaussian densities, Pk(x) -'~ N(t~k, ~k), are not indexed by the state, i. In this light, tied mixtures can be seen as a particular example of the general technique of tying to reduce the number of model parameters that must be trained [3]. "Tied mixtures" and "semi-continuous HMMs" are used in the literature to refer to HMM distributions of the …

Content truncated to 500 words