yolov5目标检测模型的知识蒸馏（基于响应的蒸馏）

Last update: Dec 04, 2022

Overview

代码地址：

https://github.com/Sharpiless/yolov5-knowledge-distillation

教师模型：

python train.py --weights weights/yolov5m.pt \
        --cfg models/yolov5m.yaml --data data/voc.yaml --epochs 50 \
        --batch-size 8 --device 0 --hyp data/hyp.scratch.yaml

蒸馏训练：

python train.py --weights weights/yolov5s.pt \
        --cfg models/yolov5s.yaml --data data/voc.yaml --epochs 50 \
        --batch-size 8 --device 0 --hyp data/hyp.scratch.yaml \
        --t_weights yolov5m.pt --distill

训练参数:

--weights：预训练模型

--t_weights：教师模型权重

--distill：使用知识蒸馏进行训练

--dist_loss：l2或者kl

--temperature：使用知识蒸馏时的温度

使用《Object detection at 200 Frames Per Second》中的损失

这篇文章分别对这几个损失函数做出改进，具体思路为只有当teacher network的objectness value高时，才学习bounding box坐标和class probabilities。

实验结果：

这里假设VOC2012中新增加的数据为无标签数据（2k张）。

教师模型	训练方法	蒸馏损失	P	R	mAP50
无	正常训练	不使用	0.7756	0.7115	0.7609
Yolov5l	output based	l2	0.7585	0.7198	0.7644
Yolov5l	output based	KL	0.7417	0.7207	0.7536
Yolov5m	output based	l2	0.7682	0.7436	0.7976
Yolov5m	output based	KL	0.7731	0.7313	0.7931

参数和细节正在完善，支持KL散度、L2 logits损失和Sigmoid蒸馏损失等

1. 正常训练：

2. L2蒸馏损失：

我的公众号：

关于作者

B站：https://space.bilibili.com/470550823

CSDN：https://blog.csdn.net/weixin_44936889

AI Studio：https://aistudio.baidu.com/aistudio/personalcenter/thirdview/67156

Github：https://github.com/Sharpiless

yolov5目标检测模型的知识蒸馏（基于响应的蒸馏）

Related tags

Overview

代码地址：

教师模型：

蒸馏训练：

训练参数:

实验结果：

1. 正常训练：

2. L2蒸馏损失：

我的公众号：

关于作者

Owner

A PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-Supervised Learning Framework".

PyTorch code for our paper "Image Super-Resolution with Non-Local Sparse Attention" (CVPR2021).

Code for EMNLP2021 paper "Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training"

TensorRT examples (Jetson, Python/C++)(object detection)

PyTorch implementation of paper A Fast Knowledge Distillation Framework for Visual Recognition.

Patient-Survival - Using Python, I developed a Machine Learning model using classification techniques such as Random Forest and SVM classifiers to predict a patient's survival status that have undergone breast cancer surgery.

Semantic Segmentation in Pytorch. Network include: FCN、FCN_ResNet、SegNet、UNet、BiSeNet、BiSeNetV2、PSPNet、DeepLabv3_plus、 HRNet、DDRNet

the official code for ICRA 2021 Paper: "Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation"

Official implementation of deep-multi-trajectory-based single object tracking (IEEE T-CSVT 2021).

[CVPR 2021] MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition

Bunch of different tools which helps visualizing and annotating images for semantic/instance segmentation tasks

Face Library is an open source package for accurate and real-time face detection and recognition

CROSS-LINGUAL ABILITY OF MULTILINGUAL BERT: AN EMPIRICAL STUDY

Official NumPy Implementation of Deep Networks from the Principle of Rate Reduction (2021)

SubOmiEmbed: Self-supervised Representation Learning of Multi-omics Data for Cancer Type Classification

Repository aimed at compiling code, papers, demos etc.. related to my PhD on 3D vision and machine learning for fruit detection and shape estimation at the university of Lincoln

机器学习、深度学习、自然语言处理等人工智能基础知识总结。

Campsite Reservation Finder

A vision library for performing sliced inference on large images/small objects

Optimized Gillespie algorithm for simulating Stochastic sPAtial models of Cancer Evolution (OG-SPACE)