Pytorch extract middle layer features?

Click on the above “ Xiaobai studies vision ”, Optional plus " Star standard " or “ Roof placement ”

 Heavy dry goods , First time delivery 

source ： Machine learning algorithms and natural language processing

edit ： Recollection

https://www.zhihu.com/question/68384370

This article is only for academic sharing , If infringement , Can delete text processing

PyTorch Extract middle layer features ？

author ： Astringent intoxication
https://www.zhihu.com/question/68384370/answer/751212803

adopt pytorch Of hook The mechanism is simply implemented , Only the output conv The feature map of the layer .

import torch
from torchvision.models import resnet18
import torch.nn as nn
from torchvision import transforms


import matplotlib.pyplot as plt




def viz(module, input):
    x = input[0][0]
    # Display at most 4 Pictures 
    min_num = np.minimum(4, x.size()[0])
    for i in range(min_num):
        plt.subplot(1, 4, i+1)
        plt.imshow(x[i])
    plt.show()




import cv2
import numpy as np
def main():
    t = transforms.Compose([transforms.ToPILImage(),
                            transforms.Resize((224, 224)),
                            transforms.ToTensor(),
                            transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                                 std=[0.229, 0.224, 0.225])
                            ])


    device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")


    model = resnet18(pretrained=True).to(device)
    for name, m in model.named_modules():
        # if not isinstance(m, torch.nn.ModuleList) and
        #         not isinstance(m, torch.nn.Sequential) and
        #         type(m) in torch.nn.__dict__.values():
        #  This is only for convoluted feature map Display 
        if isinstance(m, torch.nn.Conv2d):
            m.register_forward_pre_hook(viz)
    img = cv2.imread('/Users/edgar/Desktop/cat.jpeg')
    img = t(img).unsqueeze(0).to(device)
    with torch.no_grad():
        model(img)


if __name__ == '__main__':
    main()

The printed feature map looks like this , The characteristic diagrams of the first and fourth layers are taken .

author ： Yuan Kun
https://www.zhihu.com/question/68384370/answer/419741762

It is recommended to use hook, Without changing the network forward Function to extract the required features or gradients , In the call phase module Use to obtain the required gradient or feature .

inter_feature = {}
 inter_gradient = {}
 def make_hook(name, flag):
     if flag == 'forward':
         def hook(m, input, output):
             inter_feature[name] = input
         return hook
     elif flag == 'backward':
         def hook(m, input, output):
             inter_gradient[name] = output
         return hook
     else:
         assert False
m.register_forward_hook(make_hook(name, 'forward'))
m.register_backward_hook(make_hook(name, 'backward'))

It can achieve the function of similar hook in forward calculation and reverse calculation , Intermediate variables have been placed in inter_feature and inter_gradient.

output = model(input)  # achieve intermediate feature
loss = criterion(output, target)
loss.backward()  # achieve backward intermediate gradients

Finally, it can be released according to the demand hook.

hook.remove()

author ： Luo Yicheng
https://www.zhihu.com/question/68384370/answer/263120790

Extracting intermediate features refers to taking the intermediate weights Bring it up ? Wouldn't it be good if you didn't directly access that matrix ? pytorch When saving parameters , In fact, it is for all weights bias And so on, and then put them in a dictionary . Or you see state_dict.keys(), Find the corresponding key Just take it out .

Then it's a strange question to use it carefully ..

Even with modules Below class, You save the model because of your activation function The above itself has no parameters , So it will not be saved . Otherwise, you can try in Sequential Inside the handle relu Switch to sigmoid, You can still save what you saved before state_dict to load Go back .

It can't be said to be used with caution functional Well , I think other settings should be saved separately ( Suppose you take these as super parameters )

Interest related : to pytorch I mentioned PR

The good news ！

Xiaobai learns visual knowledge about the planet

Open to the outside world

 download 1：OpenCV-Contrib Chinese version of extension module 

 stay 「 Xiaobai studies vision 」 Official account back office reply ： Extension module Chinese course , You can download the first copy of the whole network OpenCV Extension module tutorial Chinese version , Cover expansion module installation 、SFM Algorithm 、 Stereo vision 、 Target tracking 、 Biological vision 、 Super resolution processing and other more than 20 chapters .


 download 2：Python Visual combat project 52 speak 
 stay 「 Xiaobai studies vision 」 Official account back office reply ：Python Visual combat project , You can download, including image segmentation 、 Mask detection 、 Lane line detection 、 Vehicle count 、 Add Eyeliner 、 License plate recognition 、 Character recognition 、 Emotional tests 、 Text content extraction 、 Face recognition, etc 31 A visual combat project , Help fast school computer vision .


 download 3：OpenCV Actual project 20 speak 
 stay 「 Xiaobai studies vision 」 Official account back office reply ：OpenCV Actual project 20 speak , You can download the 20 Based on OpenCV Realization 20 A real project , Realization OpenCV Learn advanced .


 Communication group 

 Welcome to join the official account reader group to communicate with your colleagues , There are SLAM、 3 d visual 、 sensor 、 Autopilot 、 Computational photography 、 testing 、 Division 、 distinguish 、 Medical imaging 、GAN、 Wechat groups such as algorithm competition （ It will be subdivided gradually in the future ）, Please scan the following micro signal clustering , remarks ：” nickname + School / company + Research direction “, for example ：” Zhang San  +  Shanghai Jiaotong University  +  Vision SLAM“. Please note... According to the format , Otherwise, it will not pass . After successful addition, they will be invited to relevant wechat groups according to the research direction . Please do not send ads in the group , Or you'll be invited out , Thanks for your understanding ~