Eye of depth (II) -- matrix and its basic operations

One 、 The basic concept and significance of matrix and common special matrix

What is the meaning of matrix , Here is a linear transformation

Distinguish clearly ： Zero matrix 、 The difference between identity matrix and diagonal matrix

Two 、 Addition, subtraction, multiplication and properties of matrices

3、 ... and 、 Matrix multiplication and properties

Be careful ： Multiplication of matrices , You must ensure that the number of columns in the first row is equal to the number of rows in the second row
Such as ： The first matrix A by m×s, The second matrix B Then for s×n, Get the matrix C Of course m×n,C = AB
and c_ij said A Multiply the rows of the matrix by B The columns of the matrix

a key ： The multiplication of matrices does not satisfy the commutative law

Four 、 The application of matrix operation in deep learning ( primary )

4.1 Digital picture recognition （ neural network ）

One picture has 0 - 9 One of the numbers , The size is 10×10, amount to 100 A digital

1. Unfold into 100 Vector （1 * 100）
2. take （1100） Matrix times w₁（100512）, Programming （1*512） An extended matrix of
3. Yes （1*512） after relu Function processing
   
4. take relu After processing （1512） multiply w₂（51210） Generate （1*10）[ Because I want to generate 10 Number ]
5. after softmax（ The back can speak ） Handle , Generate S1…S10 A probability distribution

after w₁ and w₂ The processing of reflects a transformation of feature space

The above is for a single sample , If there are multiple pictures , Then the matrix is as follows

5、 ... and 、 Trace of matrix , The transpose of the matrix , Symmetric matrix ( Covariance matrix )

5.1 Trace of matrix

The exchange multiplication of two matrices is not equal , But the trace is the same
prove ：

5.2 The transpose of the matrix

a_ij^T=a_ji

5.3 Symmetric matrix

The characteristic of symmetric matrix is : Its elements are equivalent with the diagonal as the axis of symmetry. Obviously, both the identity matrix and the diagonal matrix are symmetric matrices

Covariance matrix
Covariance matrix is a symmetric matrix