Ml8 self study notes LDA principle formula derivation

Linear discriminant analysis （LDA）（ Supervised ）

purpose ： Dimensionality reduction in data preprocessing , Classification task
The goal is ： Project the feature space to a smaller dimension K In dimensional subspace . At the same time, keep distinguishing category information .
What we care about is how to make the data more distinct .

The core point is ,w How to solve , Is how to project .

It is divided into Requirements between and within categories .

Calculate the average value of each sample

Find the mean value after projection . To calculate the mean value after projecting each data is equivalent to projecting the previous mean value .

Optimization objectives ：J（w） The bigger the better

Want to maximize J（w）, Projection to x The axis will be projected to y Larger shaft , At that time, it will be projected to x The effect of shaft classification is not good . So I put forward a concept ： Hash value . You want the hash value to be the smallest after the sample mapping . The consideration becomes ,J(w) The bigger the better ,S The smaller the better. .

Integrate the above two considerations into the objective function . Obtain the in class scatter matrix .

Interclass scatter matrix . Get the final objective function .

solve . take S_wⁱS_B See as A, namely Aw=λw,w Is the eigenvector .