Semantic segmentation ｜ learning record (2) transpose convolution

Semantic segmentation ｜ Learning record （2） Transposition convolution

Tips ： come from up Lord thunderbolt Wz, I'm just taking study notes , Original video

---

Preface

Many later networks need to use transpose convolution , So let's first introduce the related concepts of transpose convolution
The transpose convolution part mainly introduces ：

• What is transposition convolution
• Transpose convolution operation steps
• Transpose convolution common parameters
• Transpose convolution instance

One 、 What is transposition convolution ？

Transposition convolution （Transposed Convolution）, It mainly plays a On the sampling The role of .
Let's first look at the one on the left Ordinary convolution ,padding = 0,strides=1(conv), The effect is achieved through a 3 * 3 Convolution kernel , a sheet 5 * 5 The picture of becomes a 2*2 Pictures of the .
Let's look at the transposed convolution on the right ,padding = 0,strides=1(transposed conv), Be careful ️ Inside padding refer to Whether the corresponding graph after up sampling is expanded compared with the original graph ！ Here, the size of the feature layer is restored back to the size before convolution .
Emphasize a little ： Transpose convolution and Not the inverse of convolution ！ The size is restored , however The value is different from the previous value ！

A guide to convolution arithmetic for deep learning

Two 、 Operation steps of transpose convolution

• Fill between input feature map elements s-1 That's ok 、 Column 0（s It refers to the filling in the middle of the feature map 0 The number of rows or columns ）
• Fill in around the input feature map k-p-1 That's ok 、 Column 0
• The convolution kernel parameters are up and down 、 Flip left and right
• Do normal convolution （ fill 0, Step length 1）
  No padding, no strides, transposed：s=1,p=0,k=3
  

No padding, strides, transposed：s=2,p=0,k=3

Padding, strides, transposed：s=2,p=1,k=3

The width and height of the output image are calculated as follows ： With Padding, strides, transposed：s=2,p=1,k=3 Illustrate with examples .

3、 ... and 、 Examples of transpose convolution

1. After filling in the elements of the input feature map and filling around the input feature map, we get the leftmost one feature map As input for the next step （ Ignore paranoia bias）
2. Next, we will go up and down the convolution kernel parameters 、 Flip left and right
3. Then carry out the operation of ordinary convolution
   Only this and nothing more , We have completed the operation of transpose convolution
   But why do you do this ？ Further explanation will be given later .

PyTorch The official information about transpose convolution related parameters is as follows ：

Four 、 Another in-depth exploration example of transpose convolution

First, let's look at the ordinary convolution operation , I won't go over it here .

The convolution form above is what we often see , But the real computer does not slide window by window when calculating , The disadvantage of this is that it is too inefficient ！
Instead, take the following convolution kernel Equivalent matrix ：

• When the convolution kernel is in the upper left corner , Then build a form like red matrix
• Empathy , Slide to that position , Build the corresponding fill 0 Matrix
• Then the input characteristic graph is combined with the convolution equivalent matrix , Get the output characteristic graph
  The final convolution effect is the same as the sliding window effect .
  
  We will enter feature map Flattening ！ Get the picture below I
  
  Next, it will be equivalent kernel Convolution kernel is also flattened , Get the picture below C
  

I and C Multiply , Get the output characteristic graph O（ This O That is, output the result of flattening the feature layer ）

Next, we will turn the above process ： We know O and C, Can we push back the matrix I Well ？ The answer is No , In other words Convolution is not reversible , The reason lies in ： The condition that a matrix has an inverse matrix is this The matrix must be a square matrix , This condition is not satisfied here .
Although it cannot be pushed back I, But we can get one and the original I A matrix of the same size P. And then pass by reshape, We got one 4*4 A graph of the same size as before convolution , This is what was said before , Transpose convolution is actually an upsampling process .

Here, let's change another form to do the reverse process ：
We will O Restore meeting 2*2 The characteristic layer of
then $C^T$ Each column becomes a 2 * 2 Equivalent matrix of , altogether 16 individual
O And 16 individual C The decomposed small matrices are multiplied in turn

In the above process, we find every small white matrix sum of transpose convolution O The effect of multiplication and green convolution kernel sliding on the characteristic graph is equivalent . This green convolution sum is also called convolution kernel Kernel Turn up, down, left and right , This is the calculation principle of transpose convolution .
Here I believe you have a deeper understanding of transpose convolution ！