Huawei ilearning AI mathematics foundation course notes

Huawei iLearning AI Mathematical basis Course note

Course link AI Mathematical basis

One . linear algebra

1. AI At the heart of , Realize the deep learning method through matrix representation , Unstructured data –> matrix (Matrix)& vector (Vector)

2. tensor (Tensor):TensorFlow,Pytorch An important part of the equal depth learning framework , Many operations and model optimization in deep learning are based on tensor complete
   

3. In linear classifier , Application of matrix :
   

Purpose : Judge whether a figure is a cat , vehicle or people ( Scores are calculated to be similar to the scores of the three objects )

technological process : Graph processing ( Expand image to column vector )–> Solve the weight matrix & Bias matrix --> Calculation y

The formula :$y=w\ast x+b(w:powerheavyMomentfrontx:exhibitionopenhave\; toToOfThat\text{'}s\; okorColumntowardsThe\; amountb:partialSet\; upMomentfronty:junctionfruit)$

4. A matrix in space – Corresponding –> A transformation , Such a matrix is called Transformation matrix

   linear algebra = Research Spatial transformation & Vector motion Science of ,

   Both of them are realized by linear transformation

   The problems of design in linear algebra are all linear transformations or Linear mapping , The meet :

   

   Vector multiplication can enhance image or speech data eg. Conduct translation , rotate , The zoom

5. Orthogonal matrix :
   $$A^T\ast A=A\ast A^T=I,namely{A}^T=A^{-1}$$

   • The row vector and class vector of an orthogonal matrix are mutually orthogonal unit vectors

   • vector * Orthogonal matrix <=> Rotate the vector , without Telescopic or Spatial mapping effect

   • application : Orthogonal initialization ; The inverse process of orthogonal matrix –> Find matrix transposition ; Matrix decomposition

     ![[ Failed to transfer the external chain picture , The origin station may have anti-theft chain mechanism , It is suggested to save the pictures and upload them directly (img-LzLCz8mN-1637857412802)( Huawei AI Training camp .assets/image-20211124182031622.png)]](https://img-blog.csdnimg.cn/deca0ec6679e4883bbc8e75d61a93ed1.png?x-oss-process=image/watermark,type_ZHJvaWRzYW5zZmFsbGJhY2s,shadow_50,text_Q1NETiBAU3RhY2tDaGFu,size_20,color_FFFFFF,t_70,g_se,x_1

6. Diagonal matrix

   • 

   • solve and / Bad / product / power <=> Of the main diagonal element and / Bad / product / power

   • Seeking inverse

     

   • application : Due to the small amount of calculation , In machine learning, some matrices are often restricted to diagonal matrices –> Reduce the cost of Computing

7. determinant : The square array – Mapping to –> Scalar A function of , Write it down as det(A)

   • determinant <=> Directed area of matrix or Volume Promotion of ,or

     n In the space of Virgil , Describes the linear Centennial pairing " Volume " Impact

   • Value of determinant == The product of matrix eigenvalues

   • Absolute value of determinant => After the measurement matrix is multiplied Space expansion or Reduction multiple

   • Determinant plus minus => Represents the orientation of space

   • application : Find the eigenvalues of a matrix , Solving linear equations

8. The eigenvalue & Eigenvector ( Only square array ) And Matrix eigendecomposition

   • Transformation matrix A–>Aa,a Only scaling transformation occurs , No rotation or Projection effect –> be-all a* Is the eigenvector , Expansion ratio ( Similarity ratio ) Eigenvalue
   • features
     decompose : Matrix A– decompose –> Eigenvector & The eigenvalue , It's a matrix ( Matrix ) The most common method of decomposition
   • From the perspective of linear space ,λ The bigger it is –> The matrix is in λ Corresponding α The greater the variance on –> The more information
   • optimization In question , The larger the eigenvalue of the matrix , The greater the change of function value in the corresponding characteristic direction , In this direction , The greater the directional derivative
   • application :PCA Dimension reduction / optimization / Deal with model over fitting regularization

9. Singular value decomposition of matrix ( It is another kind of decomposition different from feature decomposition , Non square matrices can also be decomposed )

   • Singular value decomposition definition :

     

   • Graphic meaning :

     

     Rotate the matrix The zoom Projection Separated

   • application : Gray image compression
     

Two . probability theory & statistical

1. Study uncertainty

2. Poisson distribution – Poisson noise – Add Poisson noise to the image –> Image enhancement

3. Bayes' formula :

   

   Bayesian algorithm

4. Optimization problems

   • Unconstrained optimization problems

     • direct method ( The problem is complicated , When you can't write a specific expression , Iterative computation produces a sequence of points , Search for the best )
     • Analytic method ( Indirect method , Such as : Gradient descent method , Newton method , Quasi Newton method )

   • Constrained optimization problem ( First, through Lagrange multiplier or KKT Conditions It turns into an unconstrained problem , Solve again )

   • Gradient descent method :

     

     Gradient descent method is the method of finding the extreme value of convex function , For Nonconvex functions , At present, there is no effective method to determine its extreme value , It can only be treated as a convex function

3、 ... and . Experimental part : With video + Experimental Manual , The experimental answer

My job : Index , Know what it can achieve , What effect can be achieved .

When it is really possible to use , Can get the answer from the index in time !

1. Linear algebra experiment

• import numpy as np;from … import scipy as sp

• be based on numpy(Python Of ) and scipy Math module

• reshape( Matrix dimension transformation )

• Array.T( Matrix transposition )

• np.matmul(A,B) ( matrix multiplication )

• np.arrage(6);reshape(3,2) // establish 3×2 matrix

• print(A*B) // Matrix multiplication can also be realized

• np.linalg.inv(A) // Inverse matrix

• np.linalg.det(A) // Solve the value of determinant

• Learned singular value decomposition for gray image compression

• Solution of linear equation , Just one function (scipy.linalg.solve)

2. Probability theory experiment

• import numpy as np;from … import scipy as sp

• be based on numpy(Python Of ) and scipy Math module

• Calculating mean :np.mean(…)

• np.var(b) ;np.var(A,1); # The second parameter is 1, Means to calculate variance by line

• The realization effect of binomial distribution : Number of successes 、 An introduction to 、 Number of samples , And drawing

• Image plus noise ：

  • background ： Deep learning , Some new images need to be generated to train the classification model , But the new model cannot change fundamentally . such , You can add some noise
  • Lena Add Gauss directly to the grayscale image or Poisson noise , The noise ratio is optional

3. Optimization experiment

• Least square method Fit historical data
• Define polynomials ( That's the fitting function )
• Define the residual function