当前位置:网站首页>How to replace the backbone of the model
How to replace the backbone of the model
2022-07-07 03:22:00 【Master Ma】
1、 The principle of replacing the backbone network
Why can some models replace their internal modules ? The reason is simple , The module inside is regarded as a black box , Black box has input and output , So as a part of the network , There is input in front , Later, it also needs to be output to other modules , To replace the module and make the model work , The key is the matching of input and output , Let's take the model I have experimented with as an example .
The backbone network I want to replace is 3D One method of target detection is Group-Free 3d, The backbone network it uses is PointNet++, This is the part framed in red in the following figure .
And I want to use one transformer The backbone network Pointformer Replace PointNet++. First of all, let me talk about why I want to use Pointformer To replace PointNet++, Because in Pointformer In this paper , It said Pointformer This backbone network can be replaced PointNet++ To get better performance , I observed Group-Free 3d Is used in PointNet++ As a backbone network , But no one put one of them PointNet++ Replace with Pointformer, So I want to put Pointformer Replace PointNet++ See if you can improve Group-Free 3d Performance of .** therefore , I want to emphasize that , The backbone network I replaced is the one used by others , But it has not been practiced in the new model .** But I think the principle is the same , Just match the input and output .
2、 Check the settings of network parameters
First of all, according to the Pointformer The introduction of the paper ,Pointforemer It can be directly replaced without modification PointNet++ Of , But there is no guarantee that someone will use PointNet++ The number of input and output layers will be modified , So here you need to check the network settings of the input and output layers . For example, I used it in this experiment Group-Free 3d in PointNet++ The output size of the last layer of is 288, Generally, it is output 256, As shown in the figure below :
So change the output 256 by 288. If you don't use such a standardized form of code to concentrate input and output on one file , You can find the data input requirements of the first layer network and the output data format of the last layer network directly from the code of the backbone network .
such as :
Input :
The real start of data processing here is the red box below , At this time, it can be based on self.sa1() The input involved in this function and the parameters of its network are checked .
Output :
The last layer of the backbone network is self.fp2() This function , Similarly, you can find its network parameter design :
After modification, the backbone network can be replaced :
3、 Check whether the input and output match
After modifying the settings, you can replace the backbone network , Check the backbone network after replacement (Pointfomer) Whether the input and output in the new model are consistent with the original backbone network (PointNet++) The input and output of correspond to . First, check the input and output of the backbone network in the original method , This is the time to use debug The model , Here's the picture ,PointNet++ The input is :
Attention should be paid to forward Check the input size in the function , Here, for example. PointNet++ The input size of is torch.Size([6, 20000, 4]), The format is (batch_size, Number of point clouds , Vector length of point cloud ), Remember the size of this input , Replace with Pointformer We should also let Pointformer Can accept data of this size .
Next look at PointNet++ Output , Pull straight to forward Function return statement , Then set the same breakpoint to print out the contents and size .
Grasp the main output , It mainly focuses on the output of the next module after the backbone network output , For example, there are three outputs in one dictionary , Take another look at the input of the module behind the backbone network, which also needs these three outputs , As shown in the figure below :
The output of the backbone network here is :
torch.Size([6, 288, 1024])
torch.Size([6, 1024, 3])
torch.Size([6, 1024])
Also remember the size of these three data , Replace with Pointformer Then check Pointformer Does the output of match the size of these data .
summary
The first principle to remember is , The replaced backbone network has the same size of data input and output as the original backbone network . Then check the data input size of the first layer of the backbone network and the output size of the last layer of the backbone network , Modify the parameters , After modifying the parameters, replace the backbone network , Then check whether the input and output of the network match the input and output of the original backbone network . There are many details involved , Everyone has different problems , This article aims to explain the principle and some experience , It is impossible to go into every detail , In a word, record input and output , And then modify it , many debug That's it .
As for the effect of replacement , honestly , This is metaphysics , It involves a lot of things , Maybe your learning rate is not good enough , Even the accuracy of the previous method is not as good as . It is also possible that the module you added has a conflict with the following module, resulting in a decline in performance , And so on. .
reference :https://blog.csdn.net/weixin_44715117/article/details/125322327
边栏推荐
- Jerry's phonebook acquisition [chapter]
- Jerry's RTC clock development [chapter]
- HDU ACM 4578 Transformation-> Segment tree - interval change
- sshd[12282]: fatal: matching cipher is not supported: [email protected] [preauth]
- [colmap] 3D reconstruction with known camera pose
- Not All Points Are Equal Learning Highly Efficient Point-based Detectors for 3D LiDAR Point
- 杰理之电话本获取【篇】
- LAB1配置脚本
- 数学归纳与递归
- Significance and measures of source code confidentiality
猜你喜欢
2022.6.28
Flink task exit process and failover mechanism
制作(转换)ico图标
[tools] basic concept of database and MySQL installation
亚像素级角点检测Opencv-cornerSubPix
Error: could not find a version that satisfies the requirement xxxxx (from versions: none) solutions
Jericho turns on the display icon of the classic Bluetooth hid mobile phone to set the keyboard [chapter]
腾讯云原生数据库TDSQL-C入选信通院《云原生产品目录》
树莓派设置静态ip
编译常量、ClassLoader类、系统类加载器深度探析
随机推荐
Decoration design enterprise website management system source code (including mobile source code)
如何分析粉丝兴趣?
How to find file accessed / created just feed minutes ago
【Swift】学习笔记(一)——熟知 基础数据类型,编码风格,元组,主张
Another million qubits! Israel optical quantum start-up company completed $15million financing
体会设计细节
Do you know the five most prominent advantages of E-bidding?
Make (convert) ICO Icon
图形化工具打包YOLOv5,生成可执行文件EXE
c语言字符串排序
Cocos2d-x box2d physical engine compilation settings
Opencv environment, and open a local PC camera.
安装 torch 0.4.1
Flink task exit process and failover mechanism
HDU ACM 4578 Transformation->段树-间隔的变化
新标杆!智慧化社会治理
Codeforces round 264 (Div. 2) C gargari and Bishop [violence]
Cryptography series: detailed explanation of online certificate status protocol OCSP
Starting from 1.5, build a micro Service Framework -- log tracking traceid
Uniapp adaptation problem