[deep learning] image multi label classification task, Baidu paddleclas

Baidu PaddleClas

Baidu PaddleClas GitHub link ：https://github.com/PaddlePaddle/PaddleClas.
In the project PaddleClas/docs/en/advanced_tutorials/multilabel/multilabel_en.md Below the table of contents is a guide to the task of multi label classification , If the version iteration cannot be found, search multilabel Should be able to find . This article follows this guide to practice again .
The branch I downloaded is release/2.4, I sent a copy of the code and data used in this article to Baidu cloud ：https://pan.baidu.com/s/19d7dSK075Vs_KzwmhwxGzA?pwd=e22x .

Docker Environmental Science

Before pip install paddle The environment was ruined , This time directly Docker got , The graphics card V100 look for CUDA11 edition , Go to PaddleClas Path execution ：

sudo docker run --gpus all -v $PWD:/paddle --shm-size=8G --network=host -it paddlepaddle/paddle:2.1.0-gpu-cuda11.2-cudnn8 /bin/bash

establish docker Default when shm The size is 64M, So give it to shm-size=8G . An introduction to shared memory .

dockerHub：https://hub.docker.com/r/paddlepaddle/paddle/tags?page=1&name=gpu

Data preparation

Raw data ：https://lms.comp.nus.edu.sg/wp-content/uploads/2019/research/nuswide/NUS-WIDE.html

I handled it with Baidu NUS-WIDE-SCENE The data is much more comfortable , stay docker Container execution ：

cd /paddle/dataset

mkdir NUS-WIDE-SCENE
cd NUS-WIDE-SCENE
wget https://paddle-imagenet-models-name.bj.bcebos.com/data/NUS-SCENE-dataset.tar
tar -xf NUS-SCENE-dataset.tar

The final path ：

Training Train

First fix it BUG Talent ：
https://github.com/PaddlePaddle/PaddleClas/issues/2136

stay docker Container execution ：

unset GREP_OPTIONS

cd /paddle && python -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple --upgrade pip && pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple && pip install -r requirements.txt

single gpu Training ：

export CUDA_VISIBLE_DEVICES=0
python3 tools/train.py -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml

many gpu Training ：

export CUDA_VISIBLE_DEVICES=0,1,2
python3 -m paddle.distributed.launch --gpus="0,1,2" tools/train.py -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml

Training completion log ：

[2022/07/06 11:15:51] ppcls INFO: [Train][Epoch 10/10][Iter: 220/273]lr(CosineAnnealingDecay): 0.00008615, HammingDistance: 0.04837, AccuracyScore: 0.95163, MultiLabelLoss: 0.12250, loss: 0.12250, batch_cost: 0.11690s, reader_cost: 0.00017, ips: 547.48235 samples/s, eta: 0:00:06
[2022/07/06 11:15:53] ppcls INFO: [Train][Epoch 10/10][Iter: 230/273]lr(CosineAnnealingDecay): 0.00005571, HammingDistance: 0.04836, AccuracyScore: 0.95164, MultiLabelLoss: 0.12258, loss: 0.12258, batch_cost: 0.11685s, reader_cost: 0.00017, ips: 547.68878 samples/s, eta: 0:00:05
[2022/07/06 11:15:54] ppcls INFO: [Train][Epoch 10/10][Iter: 240/273]lr(CosineAnnealingDecay): 0.00003187, HammingDistance: 0.04824, AccuracyScore: 0.95176, MultiLabelLoss: 0.12239, loss: 0.12239, batch_cost: 0.11688s, reader_cost: 0.00017, ips: 547.55123 samples/s, eta: 0:00:03
[2022/07/06 11:15:55] ppcls INFO: [Train][Epoch 10/10][Iter: 250/273]lr(CosineAnnealingDecay): 0.00001466, HammingDistance: 0.04826, AccuracyScore: 0.95174, MultiLabelLoss: 0.12251, loss: 0.12251, batch_cost: 0.11687s, reader_cost: 0.00017, ips: 547.61131 samples/s, eta: 0:00:02
[2022/07/06 11:15:56] ppcls INFO: [Train][Epoch 10/10][Iter: 260/273]lr(CosineAnnealingDecay): 0.00000406, HammingDistance: 0.04826, AccuracyScore: 0.95174, MultiLabelLoss: 0.12253, loss: 0.12253, batch_cost: 0.11674s, reader_cost: 0.00017, ips: 548.24794 samples/s, eta: 0:00:01
[2022/07/06 11:15:57] ppcls INFO: [Train][Epoch 10/10][Iter: 270/273]lr(CosineAnnealingDecay): 0.00000006, HammingDistance: 0.04833, AccuracyScore: 0.95167, MultiLabelLoss: 0.12271, loss: 0.12271, batch_cost: 0.11677s, reader_cost: 0.00017, ips: 548.08781 samples/s, eta: 0:00:00
[2022/07/06 11:15:58] ppcls INFO: [Train][Epoch 10/10][Avg]HammingDistance: 0.04835, AccuracyScore: 0.95165, MultiLabelLoss: 0.12271, loss: 0.12271
[2022/07/06 11:16:00] ppcls INFO: [Eval][Epoch 10][Iter: 0/69]MultiLabelLoss: 0.09744, loss: 0.09744, HammingDistance: 0.03527, AccuracyScore: 0.96473, batch_cost: 2.53691s, reader_cost: 2.42421, ips: 100.91014 images/sec
[2022/07/06 11:16:05] ppcls INFO: [Eval][Epoch 10][Iter: 10/69]MultiLabelLoss: 0.12671, loss: 0.12671, HammingDistance: 0.05005, AccuracyScore: 0.94995, batch_cost: 0.38076s, reader_cost: 0.24564, ips: 672.34270 images/sec
[2022/07/06 11:16:10] ppcls INFO: [Eval][Epoch 10][Iter: 20/69]MultiLabelLoss: 0.11945, loss: 0.11945, HammingDistance: 0.04848, AccuracyScore: 0.95152, batch_cost: 0.49853s, reader_cost: 0.36578, ips: 513.50869 images/sec
[2022/07/06 11:16:15] ppcls INFO: [Eval][Epoch 10][Iter: 30/69]MultiLabelLoss: 0.12125, loss: 0.12125, HammingDistance: 0.04789, AccuracyScore: 0.95211, batch_cost: 0.47429s, reader_cost: 0.34168, ips: 539.75512 images/sec
[2022/07/06 11:16:20] ppcls INFO: [Eval][Epoch 10][Iter: 40/69]MultiLabelLoss: 0.11817, loss: 0.11817, HammingDistance: 0.04819, AccuracyScore: 0.95181, batch_cost: 0.49539s, reader_cost: 0.36272, ips: 516.76400 images/sec
[2022/07/06 11:16:24] ppcls INFO: [Eval][Epoch 10][Iter: 50/69]MultiLabelLoss: 0.10450, loss: 0.10450, HammingDistance: 0.04773, AccuracyScore: 0.95227, batch_cost: 0.47807s, reader_cost: 0.34755, ips: 535.48813 images/sec
[2022/07/06 11:16:30] ppcls INFO: [Eval][Epoch 10][Iter: 60/69]MultiLabelLoss: 0.11237, loss: 0.11237, HammingDistance: 0.04781, AccuracyScore: 0.95219, batch_cost: 0.49771s, reader_cost: 0.36855, ips: 514.35504 images/sec
[2022/07/06 11:16:32] ppcls INFO: [Eval][Epoch 10][Avg]MultiLabelLoss: 0.12195, loss: 0.12195, HammingDistance: 0.04765, AccuracyScore: 0.95235
[2022/07/06 11:16:32] ppcls INFO: [Eval][Epoch 10][best metric: 0.05279040170066246]
[2022/07/06 11:16:32] ppcls INFO: Already save model in ./output/MobileNetV1/epoch_10
[2022/07/06 11:16:33] ppcls INFO: Already save model in ./output/MobileNetV1/latest

assessment Evaluation

assessment Evaluation:

python tools/eval.py     -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml     -o Arch.pretrained="./output/MobileNetV1/best_model"

journal ：

[2022/07/06 11:37:38] ppcls INFO: train with paddle 2.1.0 and device CUDAPlace(0)
W0706 11:37:38.849289  1058 device_context.cc:404] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0706 11:37:38.854460  1058 device_context.cc:422] device: 0, cuDNN Version: 8.1.
[2022/07/06 11:37:45] ppcls INFO: [Eval][Epoch 0][Iter: 0/69]MultiLabelLoss: 0.11629, loss: 0.11629, HammingDistance: 0.04096, AccuracyScore: 0.95904, batch_cost: 3.46316s, reader_cost: 2.33148, ips: 73.92095 images/sec
[2022/07/06 11:37:48] ppcls INFO: [Eval][Epoch 0][Iter: 10/69]MultiLabelLoss: 0.14044, loss: 0.14044, HammingDistance: 0.05525, AccuracyScore: 0.94475, batch_cost: 0.45712s, reader_cost: 0.32888, ips: 560.03236 images/sec
[2022/07/06 11:37:54] ppcls INFO: [Eval][Epoch 0][Iter: 20/69]MultiLabelLoss: 0.13331, loss: 0.13331, HammingDistance: 0.05347, AccuracyScore: 0.94653, batch_cost: 0.51752s, reader_cost: 0.38610, ips: 494.66824 images/sec
[2022/07/06 11:37:58] ppcls INFO: [Eval][Epoch 0][Iter: 30/69]MultiLabelLoss: 0.14082, loss: 0.14082, HammingDistance: 0.05283, AccuracyScore: 0.94717, batch_cost: 0.48707s, reader_cost: 0.35611, ips: 525.59345 images/sec
[2022/07/06 11:38:04] ppcls INFO: [Eval][Epoch 0][Iter: 40/69]MultiLabelLoss: 0.13737, loss: 0.13737, HammingDistance: 0.05317, AccuracyScore: 0.94683, batch_cost: 0.50345s, reader_cost: 0.37316, ips: 508.48973 images/sec
[2022/07/06 11:38:08] ppcls INFO: [Eval][Epoch 0][Iter: 50/69]MultiLabelLoss: 0.12236, loss: 0.12236, HammingDistance: 0.05288, AccuracyScore: 0.94712, batch_cost: 0.48819s, reader_cost: 0.35907, ips: 524.38639 images/sec
[2022/07/06 11:38:14] ppcls INFO: [Eval][Epoch 0][Iter: 60/69]MultiLabelLoss: 0.13099, loss: 0.13099, HammingDistance: 0.05294, AccuracyScore: 0.94706, batch_cost: 0.50024s, reader_cost: 0.37077, ips: 511.75699 images/sec
[2022/07/06 11:38:16] ppcls INFO: [Eval][Epoch 0][Avg]MultiLabelLoss: 0.13865, loss: 0.13865, HammingDistance: 0.05279, AccuracyScore: 0.94721

forecast Prediction Reasoning Infer

forecast Prediction Reasoning Infer：

python3 tools/infer.py -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml  -o Arch.pretrained="./output/MobileNetV1/best_model"

journal ：

W0706 11:42:24.642689  1302 device_context.cc:404] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0706 11:42:24.648531  1302 device_context.cc:422] device: 0, cuDNN Version: 8.1.
[{
    'class_ids': [6, 13, 17, 23, 30], 'scores': [0.99138, 0.83019, 0.5909, 0.99387, 0.91533], 'file_name': './deploy/images/0517_2715693311.jpg', 'label_names': []}]

images/0517_2715693311.jpg This picture ：

Print out the data NUS_labels.txt No 6, 13, 17, 23, 30 That's ok （ from 0 Start counting ）： sed -n '7p;14p;18p;24p;31p' NUS_labels.txt （ from 1 Start counting ）, That is, the five categories obtained by the model ：

clouds
lake
ocean
sky
water

And what is the actual label of this picture , Use cat multilabel_train_list.txt |grep 0517_2715693311 obtain ：

0517_2715693311.jpg     0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,1,0,0,1,0,0,0,1,0,0

That is to say 7 14 16 24 27 31 （ from 1 Start counting ）,sed -n '7p;14p;16p;24p;27p;31p' NUS_labels.txt ：

clouds
lake
mountain
sky
sunset
water

The effect on this picture is average .

The export model Export model

Official website Introduced some conversion paddle Of the product ：

python3 tools/export_model.py \
    -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml \
    -o Arch.pretrained="./output/MobileNetV1/best_model"

cd ./deploy

python3 python/predict_cls.py \
     -c configs/inference_multilabel_cls.yaml

And I want to turn to onnx, I saw Paddle2ONNX file Only then discovered , The last step is also needed , Model structure file is required ：inference.pdmodel, Model parameter file inference.pdiparams.

install ：pip install paddle2onnx onnx onnx-simplifier onnxruntime-gpu

The export model ：paddle2onnx --model_dir inference/ --model_filename inference.pdmodel --params_filename inference.pdiparams --save_file model.onnx --opset_version 10 --enable_dev_version True --enable_onnx_checker True

Parameter options

Parameters	Parameter description
–model_dir	The configuration contains Paddle Directory path of the model
–model_filename	[ Optional ] Configuration is located in the --model_dir The file name of the storage network structure
–params_filename	[ Optional ] Configuration is located in the --model_dir Name of the file under which the model parameters are stored
–save_file	Specify the directory path to save the converted model
–opset_version	[ Optional ] The configuration is converted to ONNX Of OpSet edition , At present, we support 7~15 Waiting for multiple versions , The default is 9
–enable_dev_version	[ Optional ] Whether to use the new version Paddle2ONNX（ Recommended ）, The default is False
–enable_onnx_checker	[ Optional ] Configure whether to check export as ONNX The correctness of the model , It is recommended to turn on this switch . If specified as True, The default is False
–enable_auto_update_opset	[ Optional ] Open or not opset version Automatic upgrade , When the lower version opset When cannot convert , Automatically select a higher version opset The default is True
–input_shape_dict	[ Optional ] Configure the input shape, The default is empty. ; This parameter will be removed , If it is necessary to fix Paddle Model input Shape, Please use This tool Handle
–version	[ Optional ] see paddle2onnx edition

• Use onnxruntime Verify the transformation model , Please pay attention to installing the latest version （ Minimum requirements 1.10.0）：

If you have ONNX The need for model optimization , Recommended onnx-simplifier, You can also use the following commands to optimize the model ：

python -m paddle2onnx.optimize --input_model model.onnx --output_model new_model.onnx

If you need to modify the exported model input shape , If it is changed to static shape:

python -m paddle2onnx.optimize --input_model model.onnx \
                               --output_model new_model.onnx \
                               --input_shape_dict "{'x':[1,3,224,224]}"

Comparison of input and output of different models