Network visualization: features of convolution kernel and CNN visualization (through the attention part of gradient visualization network)

The characteristics of convolution kernel

grad-cam: Visualize the attention part of the network through the gradient

         We introduce a new method of combining characteristic graphs using gradient signals , This method does not need to modify the network structure . This enables our method to be applied to existing methods based on CNN The architecture of , Including architecture for image captioning and visual question answering . For complete convolution structures ,CAM yes Grad CAM The special case of .

Backpropagation calculation with different pooling

github Example

# https://github.com/jacobgil/pytorch-grad-cam
from pytorch_grad_cam import GradCAM, ScoreCAM, GradCAMPlusPlus, AblationCAM, XGradCAM, EigenCAM, FullGrad
from pytorch_grad_cam.utils.model_targets import ClassifierOutputTarget
from pytorch_grad_cam.utils.image import show_cam_on_image
from torchvision.models import resnet50

model = resnet50(pretrained=True)
target_layers = [model.layer4[-1]]
input_tensor = # Create an input tensor image for your model..
# Note: input_tensor can be a batch tensor with several images!

# Construct the CAM object once, and then re-use it on many images:
cam = GradCAM(model=model, target_layers=target_layers, use_cuda=args.use_cuda)

# You can also use it within a with statement, to make sure it is freed,
# In case you need to re-create it inside an outer loop:
# with GradCAM(model=model, target_layers=target_layers, use_cuda=args.use_cuda) as cam:
# ...

# We have to specify the target we want to generate
# the Class Activation Maps for.
# If targets is None, the highest scoring category
# will be used for every image in the batch.
# Here we use ClassifierOutputTarget, but you can define your own custom targets
# That are, for example, combinations of categories, or specific outputs in a non standard model.
targets = [e.g ClassifierOutputTarget(281)]

# You can also pass aug_smooth=True and eigen_smooth=True, to apply smoothing.
grayscale_cam = cam(input_tensor=input_tensor, targets=targets)

# In this example grayscale_cam has only one image in the batch:
grayscale_cam = grayscale_cam[0, :]
visualization = show_cam_on_image(rgb_img, grayscale_cam, use_rgb=True)

Reference and more

arxiv : Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
Grad-CAM Principle and Implementation ( Here the author uses hook Realized )

In the highest layer , We begin to recognize textures found in objects that are similar to the trained classification of the network , Like feathers , Eyes, etc
A noteworthy observation ： Many of these filters are the same , But through some non random factors （ Usually it is 90 degree ） rotate . This means that we can find a way to make the convolution filter rotate unchanged , Thus, it is possible to compress the number of filters used in convolution by a large factor . I can see some ways to achieve this, which is an interesting research direction .
What's shocking is that , Rotate observations even for relatively advanced filters （ for example The filter in ） It was also established

Convolution kernel Visualization

A variety of visualization tools