Knowledge Distillation 7: Detailed Explanation of Knowledge Distillation Code

Code Description

Differences from yolov5-v6.1 code:

• data/hyps/hyp.scratch-low-distillation.yaml (added file, obtained by modifying hyp.scratch-low.yaml)
• utils/loss.py adds a function compute_distillation_output_loss
• train_distillation.py (added file, obtained by modifying train.py)

hyp.scratch-low-distillation.yaml

• Compared with the original hyp.scratch-low.yaml, this file has more dist hyperparameters, which can be adjusted in the range of [0,1], close to 1The network will pay more attention to the distillation loss, and if it is close to 0, it will prefer the detection loss. This grass parameter is used to balance the detection loss and the distillation loss.
• There are also corresponding weighted losses for box loss, clss loss, obj loss.

utils/loss.py

utils/loss.py add a function compute_distillation_output_loss