[deep learning] fully connected network

1. Fully connected network

All connected neural networks （Fully Connected Netural Network,FCN） Or multi-layer perceptron （Multi-Layer Perception, MLP）, It is an artificial neural network structure with relatively simple connection mode , It belongs to a kind of feedforward neural network , As long as there is an input layer 、 Hidden layer and output layer , And there can be multiple neurons in each hidden layer .MLP Network is a multifunctional learning method that can be applied to almost all tasks , Including classification 、 Return to , Even unsupervised learning .

2. How to realize full connection

As we all know, full connection is that each parameter of the input layer participates in the calculation of each parameter of the output layer . But how exactly is it achieved ？

In conclusion ： It is realized by convolution with the same size and the same number of channels as the input layer .

for instance ：

• Two dimensional characteristic graph of convolution output of full connection （feature map） Into a one-dimensional vector , in other words ： The last two rows of small balls are two fully connected layers , At the end of the last layer of convolution , Another pooling operation is performed , Output 20 individual 12x12 Image （20 Refers to the thickness of the last layer ）, Then it passes through a full connection layer and becomes 1x100 Vector （ The number of neurons in the first full connective layer is 100）

How to do it ？

• This operation is actually done with 100 individual 20x12x12 The convolution kernel of , For each input feature map , All use a kernel convolution with the same size as the image for convolution , In this way, the whole picture becomes a number , If the thickness is 20 That's it 20 After the convolution of the two kernels, add and sum . In this way, the height of a picture can be condensed into a number .
• But there are too many parameters for full connection , You think this picture has 20 individual 12x12x100 Parameters , Any layer in front , Suppose the convolution kernel is 7*7 Of , The thickness is 64, That's just 7x7x64, So now the trend is to try to avoid full connection , At present, one of the mainstream methods is global average pooling （GlobalAveragePooling）. That's the last floor feature map（ The output of the last layer of convolution ）, Directly find the average . You can train as many layers as there are categories , These ten numbers are the corresponding probability .

3. Full connection code implementation

import torch.nn as nn
fc1 = nn.Linear(320, 50) # Input 320 dimension , Output 50 Dimension's fully connected network 

Reference material ：

• https://blog.csdn.net/gongliming_/article/details/89634243