Hands on deep learning - multiple input and output channels in the convolution layer

1、 Multiple input channels

Color images may have RGB Three channels , Converting to grayscale will lose information .

Each channel has a convolution kernel , The result is the sum of the convolution results of all channels .

Expressed by formula ：

2、 Multiple output channels

No matter how many input channels there are , So far, we only use single output channel ; We can have multiple three-dimensional convolution kernels , Each core generates an output channel .

Each output channel can recognize a specific pattern ; The input channel kernel identifies and combines patterns in the input .

3、1x1 The convolution of layer

kh=kw=1 Is a popular choice , He doesn't recognize spatial patterns , It's just a fusion channel ;

1x1 The convolution layer can also be called the full connection layer ;

Convolution layers are usually ⽤ To adjust ⽹ Number of channels between layers , And control the complexity of the model .

4、 summary

The number of output channels is a super parameter of the convolution layer ;

Each input channel has an independent two-dimensional convolution kernel , All channel results are added to obtain an output channel result ;

Each output channel has an independent three-dimensional convolution kernel .

5、 Code implementation

import torch
from torch import nn
import sys
sys.path.append("..")
import d2lzh_pytorch as d2l
""" Multiple input channels 
 Realize the cross-correlation operation with multiple input channels . We just need to do cross-correlation calculation for each channel , And then through add_n Function to accumulate 
"""
def corr2d_multi_in(X, K):
 #  Along X and K Of the 0 dimension （ Channel dimension ） Calculate separately and add up 
   res = d2l.corr2d(X[0, :, :], K[0, :, :])
   for i in range(1, X.shape[0]):
        res += d2l.corr2d(X[i, :, :], K[i, :, :])
   return res

# Construct an array x And an array k To verify the output of the cross-correlation operation 
X = torch.tensor([[[0, 1, 2], [3, 4, 5], [6, 7, 8]],[[1, 2, 3], [4, 5, 6], [7, 8, 9]]])
K = torch.tensor([[[0, 1], [2, 3]], [[1, 2], [3, 4]]])
print(corr2d_multi_in(X, K))

""" Multiple output channels 
 Implement a cross-correlation function to calculate the output of multiple channels 
"""
def corr2d_multi_in_out(X, K):
 #  Yes K Of the 0 Dimension traversal , Simultaneous transmission each time ⼊X Do cross correlation calculations . All the results make ⽤stack Functions are merged in ⼀ rise 
      return torch.stack([corr2d_multi_in(X, k) for k in K])
K = torch.stack([K, K + 1, K + 2]) # Construct a convolution kernel with three channels 
print(K.shape)
# There are three channels at the output 
print(corr2d_multi_in_out(X, K))

"""1x1 Convolution layer 
 Window size is 1x1 Multichannel convolution , transport ⼊ And output have the same ⾼ And width . Each element in the output to ⾃ transport ⼊ In the ⾼ And the weight of elements in the same position on the width between different channels ᯿ Add up .
 Suppose we treat the channel dimension as a feature dimension , take ⾼ And elements in wide dimensions as data samples , that   Convolution layer ⽤ Equivalent to the full connection layer .
"""
# Use the matrix multiplication in the full connection layer to realize 1x1 Convolution .
def corr2d_multi_in_out_1x1(X, K):
   c_i, h, w = X.shape
   c_o = K.shape[0]
   X = X.view(c_i, h * w)
   K = K.view(c_o, c_i)
   Y = torch.mm(K, X) #  Matrix multiplication of full connection layer 
   return Y.view(c_o, h, w)

X = torch.rand(3, 3, 3)
K = torch.rand(2, 3, 1, 1)
Y1 = corr2d_multi_in_out_1x1(X, K)
Y2 = corr2d_multi_in_out(X, K)
print((Y1 - Y2).norm().item() < 1e-6)