18. Convolutional neural network

Convolution

The fully connected network is pytorch It is also called linear layer , When we do linear Excluding the part of the activation function ,

receptive field Feel the field

Feel the vision , Related to local . For example, when a child looks at the table , He may pay attention to the cake first , Then focus on other things , That is, pay attention to some local things .
Convolutional neural network refers to local correlation , Focus on a small piece at a time .

weight sharing Weight sharing

When the convolution kernel moves on the image , The weights are the same , Will not change with the movement .

Parameters

The weight and output in front of each layer are added together , Calculate a layer . How many lines are there between layers , Just how many parameters .
[784,256,256,256,256,10]
784×256+256×256+256×256+256×256+256×10=399872 Parameters , Each parameter uses 4 Byte to represent , in total 1599488B, about 1.53MB.
Logically speaking, for the first point , Should have 4 line , But convolutional neural network has only 2 strip , That is, only related to it is 2 strip .

Convolution operation

Convolution kernel and the number corresponding to the input graph are multiplied and then added .

Why is it called convolution

Seeking function x(τ) and h(t-τ) The area of the overlap , We can think of convolution kernel as x(τ) function , The input graph is regarded as h(τ) function , The final output graph is seen as y(τ),y It's a two-dimensional ,y The horizontal and vertical coordinates of represent the offset , Represents the amount and direction of the convolution kernel moving on the input graph .

sharpening sharpen

0	0	0	0	0
0	0	-1	0	0
0	-1	-5	-1	0
0	0	-1	0	0
0	0	0	0	0

Fuzzy blur

0	0	0	0	0
0	0	1	0	0
0	1	1	1	0
0	0	1	0	0
0	0	0	0	0

edge detection edge detect

0	0	0	0	0
0	0	1	0	0
0	1	-4	1	0
0	0	1	0	0
0	0	0	0	0

Convolutional neural networks

I(28×28) Is the input graph function ,k It's the convolution kernel ,F It's the output function .x The range is 0->26,y And the scope of 0->26.
Different kernel, Represents different observation angles , Generate different map.

kernel：

input_channels Original channel , Black and white 1, colour 3
kernel_channels Represent several kernel, For example, edge detection 、 Fuzzy etc .
stride step , Move a few spaces at a time
padding repair 0,paddin=1, Add a line around the input diagram 0, Increase the output .

multi-kernels：

x：[b,3,28,28]
b It can be understood as the number of sheets .3 yes input channels. The size is 28×28
one k：[3,3,3]
first 3 And input channels Agreement , One kernel Yes input Of 3 Channels for feature extraction . The back is the size
multi-k：[16,3,3,3]
16 It's bias , every last kernel There is an offset , Size and kernel The same quantity .
bias：[16]
Size and kernel The same quantity
out：[b,16,28,28]
28×28 It's the size , It depends on whether there is padding.

When identifying features , The first is to observe some low-level features , Such as angle 、 Edge, etc . In the middle, we usually observe some small concepts , Such as shape . High level general observation higher level characteristics . The recognition features are superimposed layer by layer .

Realize a simple two-dimensional convolutional neural network

recommend ：layer example , Will run first hooks, Run again .forward function .
It is not recommended to use forward function .

layer=nn.Conv2d(1,3,kernel_size=3,stride=1,padding=0)
x=torch.rand(1,1,28,28)

out=layer.forward(x)
print('out shape:',out.shape)

first 1 yes input channel, We assume that there is only one black-and-white picture .3 Refer to kernel The number of .
It can be inferred from this that kernel by [3,1,3,3]
forward() Complete a forward operation of convolution .

layer=nn.Conv2d(1,3,kernel_size=3,stride=1,padding=1)
x=torch.rand(1,1,28,28)

out=layer.forward(x)
print('out shape:',out.shape)

At this point, we conducted padding, The output will be larger than the previous one .

layer=nn.Conv2d(1,3,kernel_size=3,stride=2,padding=1)
x=torch.rand(1,1,28,28)

out=layer.forward(x)
print('out shape:',out.shape)

layer=nn.Conv2d(1,3,kernel_size=3,stride=2,padding=1)
x=torch.rand(1,1,28,28)

out=layer(x)
print('out shape:',out.shape)

stride It has the function of dimension reduction , The output dimension becomes smaller

layer=nn.Conv2d(1,3,kernel_size=3,stride=2,padding=1)
print('weight:',layer.weight)
print('weight shape:',layer.weight.shape)
print('bias shape:',layer.bias.shape)

Directly use the built-in tool to update the gradient .