[yolov4] matlab simulation of network target detection based on yolov4 deep learning

       YOLO To YOLOv3 when , Basically, this series has reached a climax , In many practical tasks , You'll see them all YOLOv3 The body of , And for the simpler and more complex scenarios , For example, there are no too dense targets and extremely small targets , Most of the time just use YOLOv2 that will do . except YOLO series , There are many other excellent jobs , For example, the structure is also simple RetinaNet and SSD. the latter SSD In fact, it is often seen in practical tasks , Just in terms of performance , Slightly worse than YOLOv3, Of course , This is also because SSD No follow-up upgrade , Instead, many new jobs such as RFB-Net、DSSD And so on baseline. On performance ,RetinaNet Of course, it is not inferior to YOLOv3 Of , It's just , Compare with YOLOv3,RetinaNet One of the more fatal problems of ： Too slow . The main reason for this problem is RetinaNet Use a larger output image size and a heavier detection head .

yolov4 Innovation of
1. Innovation at the input end ： Improvement of input during training , It mainly includes Mosaic Data to enhance 、cmBN、SAT Self confrontation training

2.BackBone Backbone network ： A combination of methods and techniques , Include ：CSPDarknet53、Mish Activation function 、Dropblock

3.Neck： Target detection network in BackBone And the last output layer will often insert some layers , such as Yolov4 Medium SPP modular 、FPN+PAN structure

4.Head： The anchor frame mechanism of output layer and Yolov3 identical , The main improvement is the regression box position loss function during training CIOU_Loss, And prediction box screening nms Turn into DIOU_nms

        Generally speaking , That is to say YOLO-v4 The algorithm is based on the original YOLO Based on the target detection architecture , Adopted in recent years CNN The best optimization strategy in the field , from Data processing 、 Backbone network 、 Network training 、 Activation function 、 Loss function And other aspects have been optimized to varying degrees , Although there is no theoretical innovation , But it will be welcomed by many engineers , Attempts of various optimization algorithms . The article is like Target detected trick review , The effect has achieved FPS And Precision Balanced target detection new baseline.

          yolov4 The algorithm used in the network structure , Which retains yolov3 Of head part , Modified the backbone network to CSPDarknet53, At the same time SPP（ Space Pyramid pooling ） To expand the feeling field ,PANet As neck part .

 

yolov4 stay Technical treatment Mind maps ：

1.MATLAB Source code

clc;
clear;
close all;
warning off;
addpath(genpath(pwd));

%****************************************************************************
% More about matlab and fpga Search for “fpga and matlab” Of CSDN Blog :
%matlab/FPGA Project development cooperation 
%https://blog.csdn.net/ccsss22?type=blog
%****************************************************************************
%% Download Pretrained Network
% Set the modelName from the above ones to download that pretrained model.
modelName = 'YOLOv4-coco';
model = helper.downloadPretrainedYOLOv4(modelName);
net = model.net;

%% Load Data
% Unzip the vehicle images and load the vehicle ground truth data. 
unzip vehicleDatasetImages.zip
data = load('vehicleDatasetGroundTruth.mat');
vehicleDataset = data.vehicleDataset;

% Add the full path to the local vehicle data folder.
vehicleDataset.imageFilename = fullfile(pwd, vehicleDataset.imageFilename);


rng('default')
shuffledIndices = randperm(height(vehicleDataset));
idx = floor(0.6 * length(shuffledIndices));
trainingDataTbl = vehicleDataset(shuffledIndices(1:idx), :);
testDataTbl = vehicleDataset(shuffledIndices(idx+1:end), :);

% Create an image datastore for loading the images.
imdsTrain = imageDatastore(trainingDataTbl.imageFilename);
imdsTest = imageDatastore(testDataTbl.imageFilename);
 
% Create a datastore for the ground truth bounding boxes.
bldsTrain = boxLabelDatastore(trainingDataTbl(:, 2:end));
bldsTest = boxLabelDatastore(testDataTbl(:, 2:end));

% Combine the image and box label datastores.
trainingData = combine(imdsTrain, bldsTrain);
testData = combine(imdsTest, bldsTest);

helper.validateInputData(trainingData);
helper.validateInputData(testData);

%% Data Augmentation
augmentedTrainingData = transform(trainingData, @helper.augmentData);
 

augmentedData = cell(4,1);
for k = 1:4
    data = read(augmentedTrainingData);
    augmentedData{k} = insertShape(data{1,1}, 'Rectangle', data{1,2});
    reset(augmentedTrainingData);
end
figure
montage(augmentedData, 'BorderSize', 10)

%% Preprocess Training Data
% Specify the network input size. 
networkInputSize = net.Layers(1).InputSize;
 

preprocessedTrainingData = transform(augmentedTrainingData, @(data)helper.preprocessData(data, networkInputSize));
 
% Read the preprocessed training data.
data = read(preprocessedTrainingData);

% Display the image with the bounding boxes.
I = data{1,1};
bbox = data{1,2};
annotatedImage = insertShape(I, 'Rectangle', bbox);
annotatedImage = imresize(annotatedImage,2);
figure
imshow(annotatedImage)

% Reset the datastore.
reset(preprocessedTrainingData);

%% Modify Pretrained YOLO v4 Network

rng(0)
trainingDataForEstimation = transform(trainingData, @(data)helper.preprocessData(data, networkInputSize));
numAnchors = 9;
[anchorBoxes, meanIoU] = estimateAnchorBoxes(trainingDataForEstimation, numAnchors);

% Specify the classNames to be used in the training.
classNames = {'vehicle'};


[lgraph, networkOutputs, anchorBoxes, anchorBoxMasks] = configureYOLOv4(net, classNames, anchorBoxes, modelName);

%% Specify Training Options
numEpochs = 90;
miniBatchSize = 4;
learningRate = 0.001;
warmupPeriod = 1000;
l2Regularization = 0.001;
penaltyThreshold = 0.5;
velocity = [];

%% Train Model
if canUseParallelPool
   dispatchInBackground = true;
else
   dispatchInBackground = false;
end

mbqTrain = minibatchqueue(preprocessedTrainingData, 2,...
        "MiniBatchSize", miniBatchSize,...
        "MiniBatchFcn", @(images, boxes, labels) helper.createBatchData(images, boxes, labels, classNames), ...
        "MiniBatchFormat", ["SSCB", ""],...
        "DispatchInBackground", dispatchInBackground,...
        "OutputCast", ["", "double"]);



% Convert layer graph to dlnetwork.
net = dlnetwork(lgraph);

% Create subplots for the learning rate and mini-batch loss.
fig = figure;
[lossPlotter, learningRatePlotter] = helper.configureTrainingProgressPlotter(fig);

iteration = 0;
% Custom training loop.
for epoch = 1:numEpochs
      
    reset(mbqTrain);
    shuffle(mbqTrain);
    
    while(hasdata(mbqTrain))
        iteration = iteration + 1;
       
        [XTrain, YTrain] = next(mbqTrain);
        
        % Evaluate the model gradients and loss using dlfeval and the
        % modelGradients function.
        [gradients, state, lossInfo] = dlfeval(@modelGradients, net, XTrain, YTrain, anchorBoxes, anchorBoxMasks, penaltyThreshold, networkOutputs);

        % Apply L2 regularization.
        gradients = dlupdate(@(g,w) g + l2Regularization*w, gradients, net.Learnables);

        % Determine the current learning rate value.
        currentLR = helper.piecewiseLearningRateWithWarmup(iteration, epoch, learningRate, warmupPeriod, numEpochs);
        
        % Update the network learnable parameters using the SGDM optimizer.
        [net, velocity] = sgdmupdate(net, gradients, velocity, currentLR);

        % Update the state parameters of dlnetwork.
        net.State = state;
        
        % Display progress.
        if mod(iteration,10)==1
            helper.displayLossInfo(epoch, iteration, currentLR, lossInfo);
        end
            
        % Update training plot with new points.
        helper.updatePlots(lossPlotter, learningRatePlotter, iteration, currentLR, lossInfo.totalLoss);
    end
end

% Save the trained model with the anchors.
anchors.anchorBoxes = anchorBoxes;
anchors.anchorBoxMasks = anchorBoxMasks;

save('yolov4_trained', 'net', 'anchors');

%% Evaluate Model
confidenceThreshold = 0.5;
overlapThreshold = 0.5;

% Create a table to hold the bounding boxes, scores, and labels returned by
% the detector. 
numImages = size(testDataTbl, 1);
results = table('Size', [0 3], ...
    'VariableTypes', {'cell','cell','cell'}, ...
    'VariableNames', {'Boxes','Scores','Labels'});

% Run detector on images in the test set and collect results.
reset(testData)
while hasdata(testData)
    % Read the datastore and get the image.
    data = read(testData);
    image = data{1};
    
    % Run the detector.
    executionEnvironment = 'auto';
    [bboxes, scores, labels] = detectYOLOv4(net, image, anchors, classNames, executionEnvironment);
    
    % Collect the results.
    tbl = table({bboxes}, {scores}, {labels}, 'VariableNames', {'Boxes','Scores','Labels'});
    results = [results; tbl];
end

% Evaluate the object detector using Average Precision metric.
[ap, recall, precision] = evaluateDetectionPrecision(results, testData);

% The precision-recall (PR) curve shows how precise a detector is at varying 
% levels of recall. Ideally, the precision is 1 at all recall levels.

% Plot precision-recall curve.
figure
plot(recall, precision)
xlabel('Recall')
ylabel('Precision')
grid on
title(sprintf('Average Precision = %.2f', ap))

%% Detect Objects Using Trained YOLO v4
reset(testData)
data = read(testData);

% Get the image.
I = data{1};

% Run the detector.
executionEnvironment = 'auto';
[bboxes, scores, labels] = detectYOLOv4(net, I, anchors, classNames, executionEnvironment);

% Display the detections on image.
if ~isempty(scores)
    I = insertObjectAnnotation(I, 'rectangle', bboxes, scores);
end
figure
imshow(I)

2.yolov4 Simulation effect

 

 

 

resources