ML15 neural network (1)

neural network - Image classification - Computer vision

Images , dimension

Challenge

Routine routine

K- a near neighbor

choose 3 There are two triangles and a square around , So he belongs to the triangle ; choice 5 when , There are two triangles around him , Three squares , So it belongs to the box .

Calculation process

Algorithm analysis

Available data samples

How to make the image “ distance ” Calculation .
Directly subtract pixels , Then sum all pixels .

Results check . Take the first act , The following image is to find a similar image , However, the result is not particularly accurate , This problem will be discussed later .

Cross validation with super parameters

L2 It's European distance ,L1 It's the maharton distance . How to set the distance ？

Select parameters .
If the test set is used every time to check whether the parameters are reasonable , That will waste a lot of test sets , Test sets are very valuable .

So choose cross validation .
For the first time 1234 As a training ,5 As a test , The second time with 1245 As a training ,3 As a test . Such cross validation , Will all （5 Time ） The result processing . For example, average , In this way, multiple verification results for a parameter can be obtained

The horizontal axis hi Parameter values , The vertical axis is the accuracy .


For offsetting the image 、 Masking and graying , Will be the same as the original image L2 distance . But in fact, these pictures are different

Linear function

about x Feature weighting w Get the score of each category （ There are ten categories ）.

about 32323 That is, the cat picture is 3232 Dimensional , Color images . The product is 3072, Expand the pixels into a column , Convert the image to 3072 That's ok 1 Column vector , For classification results , Is a vector with ten rows and one column , Each row is for the probability of taking the current classification . For this column of pixels （ Equivalent to the characteristics ）, Each one needs to add a weight , So every corresponding result needs 3072 Multiply weights by features , Because there are ten results . So the weight matrix is 103072 Of .

The eigenvectors are 3*4 Of , That is, it should be divided into three categories , The attributes of the image are 4 individual , The matrix w,x Multiply by correspondence plus b You can get the score value of each category .

It is equivalent to drawing the dividing line （ Linear function ）.

But for the cat image just now , His score is higher if he belongs to the dog , Obviously, there is an error , How can we improve it .

Loss function

Take a look at the difference between the score of the current category and that of other categories . Use the loss function formula , among 1 Is the tolerance of the difference is 1, Under the current definition , It must be less than 1 Can there be no loss . A large loss value indicates that the prediction result is not good .

If the predicted value is in the green area , Is greater than the red range , So the prediction result is not very good , The loss value is relatively large , Fall in the red section , It shows that the effect is still good , Fall in the black section , It indicates that the loss value is relatively small .

Regularization penalty term

For two different models , The same loss function value is obtained . But it's actually different , These two models pay different attention to each attribute .

The regularized penalty term is the penalty weight . Square each term of the weight and then sum , Select the one with small result .

softmax classifier

SVM The output is the score . Can you output a probability ？
Just use softmax classifier . Through the score mapping to probability .

First, normalize , Score points belonging to each category e^score Calculation , Get the median value , And then Standardization , Put the current score （24.5） Divide the middle value of the score by the middle value of the score belonging to each category （24.5,164,0.18）, Get the probability of belonging to the current category .
But the maximum probability of this cat image is attributed to the car , So we have to calculate the loss value , Calculate the loss value using the score value currently belonging to the correct category , In this case, we use 0.13 Calculated , Because I want to score lower , The higher the loss , The closer the score is to one （ The higher the ）, The smaller the loss , So choose log The function is just right , Because the calculated values are basically negative , More to negate . as follows 2 chart .

Comparison of two loss functions

If the scores for each category are similar, the calculated loss value is relatively small or even close to zero , So the first loss function is not good enough . But the second will be done first EXP mapping , The value will be mapped first , There will be no defect of the first kind .

The problem to be solved by the objective function - optimization problem

Downhill problem , Want to find J(θ) minimum value , Use the gradient descent method to find J(θ), How many are there in each step θ value , Combine several values to find the direction of the fastest decline . Seek a new direction every step

Optimization problems

Forward propagation

It is the process from input data to loss value （x->loss）. Through forward propagation to get loss value , Then optimize the parameter value through back propagation .

epoch The value is equivalent to the number of iterations of all images ,batch It's a choice （ If say ）64 The number of complete forward and back propagation of images . It can be seen that the actual situation may also increase the loss value after each iteration , But the overall decline .

Every time you update , Add an updated limit . It is similar to the gradient descent , What is the step length of each walk .（0.001 perhaps 0.0001 Such a small one ）

Back propagation

An example of back propagation .f The value of is determined by the following equation , Each parameter can be obtained by partial derivation （x,y,z） How much contribution to the result . In back propagation, it is required to figure out how much influence each weight parameter has on the result .

The gradient is from L to z Again from z to x,y. Look at the red 、 Transmission path .

Illustrate with examples ：z=x*y, seek x relative z The degree of contribution is to seek z Yes x Partial derivative of , So that is y; seek y relative z The degree of contribution is to seek z Yes y Partial derivative of , So that is z, Equivalent to the feeling of exchange .

The next part officially starts with Neural Networks .