Numpy quick start (V) -- Linear Algebra

• This article is about NumPy The last in the quick start series .
• The matrix mentioned in this article refers to np.array object , Instead of np.matrix object , The latter is no longer recommended .
• NumPy in , The modules related to linear algebra are np.linalg （linear algebra）, We usually abbreviate it as LA, namely

from numpy import linalg as LA

One 、@ operator

Numpy in ,@ The operator can be used to calculate the product of two matrices ：

A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])
print(A @ B)
# [[19 22]
# [43 50]]

np.matmul It can also be used to calculate the product of two matrices , But it is more recommended here @.

Two 、 Vector product and matrix product

function	effect
np.inner(a, b)	a and b It's all One dimensional array （ vector ）; Calculate the of two vectors Inner product
np.outer(a, b)	Calculate the of two vectors Exoproduct
np.kron(a, b)	Calculate the Kronecker's product
LA.matrix_power(A, n)	$A$ yes Matrix ; Calculation $A^n$

2.1 np.inner()

set up $a$ and $b$ Are the two （ Column ） vector , be np.inner(a, b) amount to $a^{\mathrm{T}}b$.

a = np.array([1, 2, 3])
b = np.array([3, 2, 1])
print(np.inner(a, b))
# 10

2.2 np.outer()

set up $a$ and $b$ Are the two （ Column ） vector , be np.outer(a, b) amount to $a{b}^{\mathrm{T}}$.

a = np.array([1, 2, 3])
b = np.array([3, 2, 1])
print(np.outer(a, b))
# [[3 2 1]
# [6 4 2]
# [9 6 3]]

2.3 np.kron()

For the definition of Kronecker product, please refer to Wikipedia. Kronecker product.

In most cases, we calculate the Kronecker product of two matrices , set up $A$ and $B$ It's two matrices , be np.kron(a, b) amount to $A\otimes B$.

A = np.array([[1, 2], [3, 4]])
B = np.array([[0, 5], [6, 7]])
print(np.kron(A, B))
# [[ 0 5 0 10]
# [ 6 7 12 14]
# [ 0 15 0 20]
# [18 21 24 28]]

2.4 LA.matrix_power()

We use nilpotent matrix to observe the effect ：

A = np.eye(4, k=1)
print(A)
# [[0. 1. 0. 0.]
# [0. 0. 1. 0.]
# [0. 0. 0. 1.]
# [0. 0. 0. 0.]]
print(LA.matrix_power(A, 2))
# [[0. 0. 1. 0.]
# [0. 0. 0. 1.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]]
print(LA.matrix_power(A, 3))
# [[0. 0. 0. 1.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]]
print(LA.matrix_power(A, 4))
# [[0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]
# [0. 0. 0. 0.]]

3、 ... and 、 determinant 、 Rank 、 trace 、 norm

function	effect
LA.det(A)	Calculation of matrix A The determinant of
LA.matrix_rank(A)	Calculation of matrix A The rank of
np.trace(A)	Calculation of matrix A The trace of
LA.norm(a, ord=None)	ord Control norm ; Calculate the norm of a vector or matrix

3.1 LA.det()

A = np.array([[1, 2], [3, 4]])
print(LA.det(A))
# -2.0000000000000004

3.2 LA.matrix_rank()

A = np.array([[1, 2], [3, 4]])
B = np.array([[1, 1], [1, 1]])
C = np.array([[0, 0], [0, 0]])
print(LA.matrix_rank(A))
print(LA.matrix_rank(B))
print(LA.matrix_rank(C))
# 2
# 1
# 0

3.3 np.trace()

A = np.identity(4)
print(np.trace(A))
# 4.0

3.4 LA.norm()

Before further explanation of usage , Let's first review the common norms of matrices and vectors ：

Matrix norm	expression
Frobenius norm	$\Vert A{\Vert }_F=(\sum _{i,j}\mid a_{ij}{\mid }^2{)}^{1/2}$
$1$ - norm	$\Vert A{\Vert }_1=\underset{j}{\max }\sum _i\mid a_{ij}\mid$
$2$ - norm	$\Vert A{\Vert }_2=\sqrt{{\lambda }_{max}(A^{H}A)}$
$\infty$ - norm	$\Vert A{\Vert }_{\infty }=\underset{i}{\max }\sum _j\mid a_{ij}\mid$
Kernel norm	$\Vert A{\Vert }_{\ast }=\mathrm{t}\mathrm{r}(\sqrt{A^{H}A})$

Vector norm	expression
$p$ norm （$1\le p<+\infty$）	$\Vert \mathbf{x}{\Vert }_p=(\sum _i\mid x_i{\mid }^p{)}^{1/p}$
$\infty$ - norm	$\Vert \mathbf{x}{\Vert }_{\infty }=\underset{i}{\max }\mid x_i\mid$

Let's take a look at ord The common value of ：

ord	Matrix norm	Vector norm
None	Frobenius norm	$2$ - norm
‘nuc’	Kernel norm	——
inf	max(sum(abs(a), axis=1))	max(abs(a))
1	max(sum(abs(a), axis=0))	sum(abs(a))
2	$2$ - norm	sum(abs(a) $\ast \ast$ 2) $\ast \ast$ (1./2)

Specific code examples will not be given , Readers can practice by themselves if necessary .

Four 、 Eigenvalues and eigenvectors

function	effect
LA.eig(A)	Computational matrix A Of The eigenvalue and Eigenvector
LA.eigh(A)	Calculation of matrix A Eigenvalues and eigenvectors （A yes Real symmetry Matrix or Hermite matrix ）
LA.eigvals(A)	Computational matrix A Of The eigenvalue
LA.eigvalsh(A)	Computational matrix A Of The eigenvalue （A yes Real symmetry Matrix or Hermite matrix ）

4.1 LA.eig()

Let's first review the relevant definitions .

Only real number fields are considered here . set up $A$ yes $n$ Square matrix , If there is a constant $\lambda$ And nonzero vectors $x$ bring

$$Ax=\lambda x$$

said $\lambda$ yes $A$ Of The eigenvalue ,$x$ yes $A$ Belong to characteristic value $\lambda$ Of Eigenvector .

Do not consider repeated situations ,$A$ There must be $n$ Eigenvalues , Write it down as ${\lambda }_1,\cdots ,{\lambda }_n$, The corresponding eigenvector is recorded as ${\mathbf{p}}_1,\cdots ,{\mathbf{p}}_n$.

Make $\mathbf{\lambda }=[{\lambda }_1,\cdots ,{\lambda }_n]$,$\mathbf{P}=[{\mathbf{p}}_1,\cdots ,{\mathbf{p}}_n]$, You know $\mathbf{\lambda }$ It's a one-dimensional array ,$\mathbf{P}$ Is a two-dimensional array . Yes $A$ Use LA.eig(A) Will return a Tuples $(\mathbf{\lambda },\mathbf{P})$, We usually use two parameters w, v Receive ：

A = np.diag((3, 2, 1))
w, v = LA.eig(A)
print(w)
# [3. 2. 1.]
print(v)
# [[1. 0. 0.]
# [0. 1. 0.]
# [0. 0. 1.]]

Combined with the above definition , We can see ：

v[:, i] yes $A$ Belong to characteristic value w[i] Eigenvector of .

Let's take another example

A = np.array([
    [1, -1, 3],
    [0, 1, 2],
    [0, 0, 2]
])
w, v = LA.eig(A)
print(w)
# [1. 1. 2.]
print(v)
# [[1.00000000e+00 1.00000000e+00 4.08248290e-01]
# [0.00000000e+00 2.22044605e-16 8.16496581e-01]
# [0.00000000e+00 0.00000000e+00 4.08248290e-01]]

It is not difficult to see from the results that $\Vert {\mathbf{p}}_i\Vert =1$, That is, each eigenvector after return v[:, i] All are Standardization the . Besides , It can also be seen from the first example , Our eigenvalue list w It is not arranged from small to large .

4.2 LA.eigh()

The list of characteristic values returned by this function w yes From small to large To arrange （ Real symmetry / The eigenvalue of Hermite matrix is always The set of real Numbers ）.

For real symmetric matrix and Hermite matrix , Of course, we can also use LA.eig Calculate , But there is no doubt that LA.eigh It's better , Because it uses real symmetry / The properties of Hermite matrix thus Speed up the calculation .

Specific code examples will not be given , Readers can practice by themselves if necessary .

4.3 LA.eigvals()

This function only returns the list of characteristic values w, And w not always It is arranged from small to large .

A = np.diag((3, 2, 1))
print(LA.eigvals(A))
# [3. 2. 1.]

4.4 LA.eigvalsh()

The list of characteristic values returned by this function w yes From small to large To arrange .

Specific code examples will not be given , Readers can practice by themselves if necessary .