ML - Speech - traditional speech model

Reference resources

Huawei cloud College

GMM

hybrid model

The hybrid model is a model composed of K A mixed distribution consisting of sub distributions , It represents the probability distribution of the observed data in the population . for example ： The model mixed by several Gaussian distributions is called Gaussian mixture model , The model with several linear models mixed together is called linear mixed model .

The hybrid model is a statistical model , Including fixed effects and random effects . In statistics , Hybrid model is a probability model that represents the existence of subgroups in a large group .

Hybrid model definition

Gaussian mixture model definition

Gaussian Mixture Model（ abbreviation GMM）. Gaussian mixture model uses Gaussian probability density function （ Normal distribution curve ） Accurately quantify things , Decompose things into several models based on Gaussian probability density function .

GMM Is an extension of a single Gaussian probability density function , It can smoothly approximate the density distribution of any shape .GMM Categories include single Gaussian models （Single Gaussian Model,SGM） And Gaussian mixture model （Gaussian Mixture Model,GMM） Two types of .

Similar clustering , According to the Gaussian probability density function （Probability Density Function,PDF） Different parameters , Every Gaussian model can be regarded as a class , Enter a sample x, You can pass PDF Calculate its value , Then a threshold is used to judge whether the sample belongs to the Gaussian model .

A random variable

A real valued single valued function representing the results of a random trial . For example, the number of passengers in the bus station at a certain time .

Discrete random variables

That is, within a certain interval, the value of variables is limited or countable . For example, the number of births in a certain area in a certain year 、 Number of deaths 、 The effective number of patients treated with a drug 、 Invalid number, etc .

Continuous random variable

That is, there are infinite variables in a certain interval , For example, the length of men in a certain area 、 Weight value, etc .

Probability density function

The probability density function of continuous random variables is a function that describes the possibility of the output value of random variables near a certain value point . The probability that the value of a random variable falls within a certain region is the integral of the probability density function in this region .

Gaussian distribution

Gaussian distribution , Also known as normal distribution , It was first obtained by dimofo in the asymptotic formula of binomial distribution . Gauss derived it from another angle when he studied the measurement error , In Mathematics 、 Probability distribution is very important in physics and engineering , Has a significant impact on many aspects of Statistics .

Gaussian distribution curve

The normal curve is bell shaped , Both ends are low , Middle high , Right and left symmetry . The greater the standard deviation , The flatter the curve is ; conversely , The thinner the curve, the higher .

Single Gaussian model

When sample data X When it is one-dimensional data , The Gaussian distribution follows the following probability density function ：

When sample data X When it is multidimensional data , The Gaussian distribution follows the following probability density function ：

Maximum likelihood method

Maximum likelihood method （Maximum Likelihood,ML） Also called maximum likelihood estimation , It is a theoretical point estimation method . Maximum likelihood estimation is a statistical method , It is used to find the parameters of the correlation probability density function of a sample set .
The basic idea ： Randomly select from the model population n After a group of sample observations , The most reasonable parameter estimator should be such that the parameter is extracted from the model n The probability of group sample observation is the highest .

Maximum likelihood estimation

Probability and likelihood

Single Gaussian model parameter learning

Solving steps ：（1） Probability density function .（2） Likelihood function .（3） Log likelihood function .（4） Find the derivative and make the equation zero .（5） solve equations .

Gaussian mixture model

Gaussian mixture model parameter learning

EM Algorithm

Maximum expectation algorithm （Expectation Maximization Algorithm）, It's an iterative algorithm , Used to contain hidden variables （Hidden Variable） Maximum likelihood estimation or maximum a posteriori probability estimation of probability parameter model .
The algorithm is Dempster,Laind,Rubun On 1977 The method of finding maximum likelihood estimation parameters proposed in , It can modify parameters from incomplete data sets MLE It is estimated that , It can be widely used to deal with defective data , Censored data , So called incomplete data with noise .

EM Algorithmic solution

The overall procedure is ：（1） Initialize parameters .（2）E step ： Expect .（3）M step ： To seek maximum , Calculate the model parameters for a new iteration .（4） Iteration to convergence .

GMM Learning steps

（1） Gaussian mixture model function
（2） Probability density function
（3） Likelihood function
（4） Log likelihood function
（5）EM Algorithmic solution

GMM Advantages and disadvantages

advantage ： Strong fitting ability , Maximize the probability of speech feature matching .
shortcoming ： Unable to process sequence , Cannot process linear or near linear data .

HMM

Markov chain case

A similar commodity A,B,C Different publicity efforts , Under the effect of advertising, customers try to buy goods for the first time A,B,C The probabilities are 0.2,0.4,0.4. Customers' purchase preferences are shown in the following table , Ask a customer 4 The probability of buying each commodity at a time .

Markov chain case solving

Markov chain

Markov chain is a discrete event random process with Markov property in Mathematics . In the process , Given current knowledge or information , The past has nothing to do with predicting the future , Only related to the current state .
At every step of the Markov chain , System according to probability distribution , Can change from one state to another , You can also keep the current state . Change of state is called transference , The probabilities associated with different state changes are called transition probabilities .

Markov chain principle

principle ： Markov chains describe a sequence of states , Each state value depends on the previous finite states . Markov chain is a sequence of random variables with Markov properties . The range of these variables , That is, the set of all their possible values , go by the name of “ The state space ”.
Positive definiteness ： Each element in the state transition matrix is called the state transition probability , Each state transition probability is positive
Finiteness ： Each row in the state transition matrix is added as 1.

Observable Markov model

The hidden Markov model （Hidden Markov Model,HMM） It is a kind of Markov chain , Its state cannot be directly observed , But it can be observed through the sequence of observation vectors , Each observation vector is expressed in various states through some probability density distribution , Each observation vector is generated by a state sequence with a corresponding probability density distribution . So Markov model is a double random process , Hidden Markov chains and random function sets with a certain number of States .

HMM describe

HMM Three major issues

Evaluation questions ： Forward algorithm 、 Backward algorithm
Decoding problem ： Dynamic programming algorithm 、Viterbi Algorithm
Learning problems ： Supervised algorithm 、 Unsupervised Baum-Welch Algorithm

HMM evaluation - Forward algorithm

The so-called evaluation problem is to calculate HMM On the likelihood ratio of a particular observation sequence （likelihood）. Given a HMM Model , Parameter is λ=(A,B) And an observation sequence O=o1o2…oT, Calculate the likelihood ratio of the observation sequence P(O|λ).
Algorithm steps ：

HMM evaluation - Backward algorithm

HMM Study - Supervised

HMM Study - Unsupervised Bauw-Welch

HMM decode - Viterbi

viterbi algorithm （Viterbi） It is a special but widely used dynamic programming algorithm , It is proposed for the shortest path problem of directed graph of fence network . Any problem described by hidden Markov model can be decoded by Viterbi Algorithm , Including today's digital communications 、 speech recognition 、 Machine translation, etc . The steps include ： initialization 、 recursive 、 End 、 Optimal path backtracking .

HMM Applications in speech recognition

（1） Forward backward algorithm calculation P(O|A), Output sequence \ Implicit sequence
（2）Baum-Welch The algorithm finds the optimal solution λ=max(P(O|A)).
（3） For input voice , use Viterbi1 The algorithm finds out which HMM The probability of the model is the greatest , The best sequence is thus obtained
（4） Combine phonemes and words according to the best sequence
（5） Form words and sentences according to language models