Up sampling deconvolution operation

In this paper, the reference ：

https://www.jianshu.com/p/b48bf190fe61

Understand transpose convolution - Alicyclic - Blog Garden

Deep parsing what is transposed convolution - Electronic enthusiast network

https://blog.csdn.net/weixin_41620490/article/details/105885663

Commonly used down sampling is through convolution and pooling , Keep reducing the image size , Reduce the sampling points of the matrix .

Upsampling is performed by deconvolution or interpolation , Constantly expand the image size , Increase the sampling points of the matrix .

Convolution itself is a feature extraction , The process of data compression .

Deconvolution is a special forward convolution , First pass the supplement according to a certain proportion 0 To expand the size of the input image , Then rotate the convolution kernel , Then forward convolution . But this deconvolution is not the inverse process of convolution , Once the convolution operation is over , It cannot be restored by deconvolution . Deconvolution should be transposed convolution more accurately .

How to understand transpose ？

Let's start with convolution .

For the convolution operation above ,

Input input by 4*4,kernel by 3*3,stride by 1,padding by 0. Then the convolution is ：

 

Mathematical derivation can be achieved by sliding kernel Calculate each value separately , But the computer calculates the value at one time .

First, convert the input into N*1, For Ben input by 16*1, as follows ：

Transform convolution kernel into 4*16 The sparse matrix of , As shown in the figure below ：

  Matrix multiplication 4*1 The process of the result is as follows ：

Constructed by convolution kernel 4*16 The sparse matrix of , bring 4*4 -> 16*1 The input of gets directly 4*1 -> 2*2 Result .

The upsampling is the opposite , We hope 2*2 -> 4*1 The input of gets 16*1 -> 4*4 Result , At this point, you need to use 16*4 Matrix , From positive 4*16 The matrix becomes 16*4 The transpose matrix of , Hence the name transpose matrix .

 

How to calculate the output size ？

First, the calculation formula of convolution output size is given ：

After transformation, we get ：

 

Exchange in and out Value , Then the size of the transposed convolution is ：

 

pytorch How to calculate transpose convolution ？

First , The code of convolution operation is as follows ：

import torch
import torch.nn as nn

model = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=2,stride=1,padding=0)
x = torch.tensor([[[1.,2.,3.],[4.,5.,6.],[7.,8.,9.]]]).unsqueeze(0)
model.weight.data = torch.tensor([[[[1.,1.],[1.,1.]]]])
model.bias.data = torch.zeros(1)

print(model(x))

The output is ：

tensor([[[[12., 16.],
          [24., 28.]]]], grad_fn=<ThnnConv2DBackward>)

  The code of transpose convolution operation is as follows ：

import torch
import torch.nn as nn

model = nn.ConvTranspose2d(in_channels=1, out_channels=1, kernel_size=3,stride=1,padding=0)
x = torch.tensor([[[12,16],[24,28]]], dtype=torch.float32).unsqueeze(0)
model.weight.data = torch.tensor([[[[1,1,1],[1,1,1],[1,1,1]]]], dtype=torch.float32)
model.bias.data = torch.zeros(1)

print(model(x))

  The output is ：

tensor([[[[12., 28., 28., 16.],
          [36., 80., 80., 44.],
          [36., 80., 80., 44.],
          [24., 52., 52., 28.]]]], grad_fn=<SlowConvTranspose2DBackward>)