当前位置:网站首页>BN folding and its quantification
BN folding and its quantification
2022-07-06 08:56:00 【cyz0202】
This paper introduces the process of quantification BN Fold ;
The following pictures are quoted from Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
BN Fold
First introduced BN Fold ;BN Folding comes from BN The implementation difference between the training phase and the inference phase of the module ;
As shown in the figure below : With conv+BN For example , chart 1 For the training stage BN, chart 2 For the inferential stage BN
It can be proved that such folding is reasonable , Pay attention to the picture 2 Medium w It refers to the convolution kernel ;
In quantification BN Fold
For quantification , Especially quantitative perception ( Pseudo quantization ), We need to keep the pseudo quantization in the training stage and the above figure 2 The quantitative implementation of the inference phase of is consistent , Therefore, for the training stage CONV+BN Fold in two steps , Here's the picture 3:
chart 3 Our design idea comes from figure 2, The basic idea is to seek Of EMA( In the picture moment And the steps before and after ) After taking it off, you should match the picture 2 Agreement ;
According to the figure 3 Shown BN Fold , You can do post training quantification or pseudo quantification of the folding in the training stage , Here's the picture 4
Code implementation
The following code block is from github distiller, There is no complete context implementation , Interested readers can read step by step
summary
- The above is a brief introduction to BN Folding and its quantification , The thought is quite ingenious , Interested readers can further read in depth according to the literature mentioned in the article , thank you
- Please correct any misunderstandings
边栏推荐
- LeetCode:劍指 Offer 42. 連續子數組的最大和
- Shift Operators
- TP-LINK 企业路由器 PPTP 配置
- TDengine 社区问题双周精选 | 第三期
- LeetCode:39. 组合总和
- Introduction to the differences between compiler options of GCC dynamic library FPIC and FPIC
- LeetCode:26. Remove duplicates from an ordered array
- Esp8266-rtos IOT development
- 多元聚类分析
- LeetCode:剑指 Offer 42. 连续子数组的最大和
猜你喜欢
Warning in install. packages : package ‘RGtk2’ is not available for this version of R
Improved deep embedded clustering with local structure preservation (Idec)
TP-LINK enterprise router PPTP configuration
Target detection - pytorch uses mobilenet series (V1, V2, V3) to build yolov4 target detection platform
LeetCode:124. Maximum path sum in binary tree
UML图记忆技巧
Alibaba cloud server mining virus solution (practiced)
[embedded] cortex m4f DSP Library
Mongodb installation and basic operation
Computer graduation design PHP Zhiduo online learning platform
随机推荐
SimCLR:NLP中的对比学习
[OC]-<UI入门>--常用控件的学习
自动化测试框架有什么作用?上海专业第三方软件测试公司安利
LeetCode:836. 矩形重叠
自定义卷积注意力算子的CUDA实现
LeetCode:498. 对角线遍历
LeetCode:214. 最短回文串
LeetCode:836. Rectangle overlap
Current situation and trend of character animation
LeetCode:剑指 Offer 48. 最长不含重复字符的子字符串
The ECU of 21 Audi q5l 45tfsi brushes is upgraded to master special adjustment, and the horsepower is safely and stably increased to 305 horsepower
[MySQL] limit implements paging
LeetCode:劍指 Offer 42. 連續子數組的最大和
LeetCode:124. 二叉树中的最大路径和
LeetCode:39. 组合总和
LeetCode:221. 最大正方形
The network model established by torch is displayed by torch viz
Mongodb installation and basic operation
Guangzhou will promote the construction of a child friendly city, and will explore the establishment of a safe area 200 meters around the school
704 binary search