Abstract: Gaussian mixture model are commonly used as probability density of observed data in Hidden Markov Models. The parameters of the Gaussian mixture model is estimated based on Maximum Likelihood principle in almost all automatic speech recognition systems. However, the value of the likelihood would monotonically increase with the increases of the number of the components. On the other hand, parameters of over-complex model could not be welltrained, and give bad results for testing data. In this paper, we present an estimation of the parameters of Gaussian mixture models based on Structural Risk Minimization principle which is the key principle of statistical learning theory. We further generalized the method for the purpose that simple model is more attractive in real-time systems. The new methodology could reduce the error rate consistently compared with baseline systems which has the same average number of components.