Paddleclas classification practice record

preface

paddleclas It integrates many excellent classification models , Use paddleclas Classify and train data sets , Record the training process , For later reference .

One 、 Prepare the dataset

Default paddle The environment is installed , Prepare as required imageNet1k Formatted data , The format is as follows ：

├── train
│   ├── n01440764
│   │   ├── n01440764_10026.JPEG
│   │   ├── n01440764_10027.JPEG
├── train.txt
...
├── val
│   ├── ILSVRC2012_val_00000001.JPEG
│   ├── ILSVRC2012_val_00000002.JPEG
├── val.txt

PaddleClas Use txt Format file specifies training set and test set , With ImageNet1k Data sets, for example , among train_list.txt and val_list.txt The format of is as follows ：

#  Each line uses " Space " Separate image path and annotation 

#  Here is  train_list.txt  Sample formats in 
train/n01440764/n01440764_10026.JPEG 0
...
#  Here is  val_list.txt  Sample formats in 
val/ILSVRC2012_val_00000001.JPEG 65

train.txt and val.txt Script generation preparation ：

import os
import cv2

classname = ["cat","dog","brid"]

img_path = ["/data/train_data/train/","/data/train_data/test/"]
file_train = open("train.txt","w")
file_test = open("val.txt","w")

for i,images_path in enumerate(img_path):
    for indx ,clsname in enumerate(classname):
        for name in os.listdir(os.path.join(images_path,clsname)):
            if i ==0:
                file_train.write("train/"+clsname+"/"+name+" "+str(indx))
                file_train.write("\n")
            else:
                file_test.write("test/" + clsname + "/" + name + " " + str(indx))
                file_test.write("\n")

file_train.close()
file_test.close()

Two 、 Configuration modification

Because it's training imageNet1k Format data , So modify ppcls/configs/ImageNet Configuration file in , Modify whichever you want to train , Here I use PPLCNet/PPLCNetV2_base.yaml, Key content modification object ：

• Pre training model address
• Training rounds
• Training picture size
• Number of classification network categories
• Learning rate
• Dataset address
• Training batch
• Data to enhance

The example is modified as follows ：

# global configs
Global:
  checkpoints: null
  pretrained_model: ./models/PPLCNetV2_base_pretrained                     #  Pre training model address 
  output_dir: ./output/                                                    #  Output address 
  device: gpu
  save_interval: 1 
  eval_during_train: True
  eval_interval: 1
  epochs: 100                                   #  Training rounds 
  print_batch_step: 10 
  use_visualdl: False
  # used for static mode and model export
  image_shape: [3, 768, 768]          #  Training picture size 
  save_inference_dir: ./inference

# model architecture
Arch:
  name: PPLCNetV2_base
  class_num: 4                              #  Number of categories 

# loss function config for traing/eval process
Loss:
  Train:
    - CELoss:
        weight: 1.0
        epsilon: 0.1
  Eval:
    - CELoss:
        weight: 1.0

Optimizer:
  name: Momentum
  momentum: 0.9
  lr:
    name: Cosine
    learning_rate: 0.1                              #  Learning rate 
    warmup_epoch: 3                           #  Preheating round 
  regularizer:
    name: 'L2'
    coeff: 0.00004

# data loader for train and eval
DataLoader:
  Train:
    dataset:
      name: ImageNetDataset                                       #  Dataset type 
      image_root: /home/cai/data/train_data/                                   #  picture root Address 
      cls_label_path: /home/cai/data/train_data/train.txt                  # txt Address 
      transform_ops:
        - DecodeImage:
            to_rgb: True
            channel_first: False
        - RandCropImage:                          
            size: 736
        - RandFlipImage:
            flip_code: 1
        - NormalizeImage:
            scale: 1.0/255.0
            mean: [0.485, 0.456, 0.406]
            std: [0.229, 0.224, 0.225]
            order: ''
    sampler:
      name: DistributedBatchSampler
      batch_size: 8                                    #  Training batch 
      drop_last: False
      shuffle: True
    loader:
      num_workers: 4
      use_shared_memory: True

  Eval:
    dataset: 
      name: ImageNetDataset
      image_root: /home/cai/data/train_data/
      cls_label_path: /home/cai/data/train_data/val.txt
      transform_ops:
        - DecodeImage:
            to_rgb: True
            channel_first: False
        - ResizeImage:
            resize_short: 768
        - CropImage:
            size: 736
        - NormalizeImage:
            scale: 1.0/255.0
            mean: [0.485, 0.456, 0.406]
            std: [0.229, 0.224, 0.225]
            order: ''
    sampler:
      name: DistributedBatchSampler
      batch_size: 8
      drop_last: False
      shuffle: False
    loader:
      num_workers: 4
      use_shared_memory: True

Infer:
  infer_imgs: docs/images/inference_deployment/whl_demo.jpg
  batch_size: 10
  transforms:
    - DecodeImage:
        to_rgb: True
        channel_first: False
    - ResizeImage:
        resize_short: 768
    - CropImage:
        size: 736
    - NormalizeImage:
        scale: 1.0/255.0
        mean: [0.485, 0.456, 0.406]
        std: [0.229, 0.224, 0.225]
        order: ''
    - ToCHWImage:
  PostProcess:
    name: Topk
    topk: 3                                    #  If the number of categories is less than 5 Words 
    class_id_map_file: ppcls/utils/label_list.txt                          #  Category label 

Metric:
  Train:
    - TopkAcc:
        topk: [1, 3]
  Eval:
    - TopkAcc:
        topk: [1, 3]

3、 ... and 、 model training

Command line run ：

python tools/train.py -c ppcls/configs/ImageNet/PPLCNetV2/PPLCNetV2_base.yaml 

The model with the best accuracy at present will be saved in output/PPLCNetV2_base/best_model.pdparams

Four 、 Model to evaluate

python3 tools/eval.py -c ppcls/configs/ImageNet/PPLCNetV2/PPLCNetV2_base.yaml -o Global.pretrained_model=output/PPLCNetV2_base/best_model

-o Global.pretrained_model=“output/PPLCNetV2_base/best_model” Specifies the path where the current best weight is located , If you specify other weights , Just replace the corresponding path .

5、 ... and 、 Model to predict

python3 tools/infer.py \
    -c ppcls/configs/ImageNet/PPLCNetV2/PPLCNetV2_base.yaml \
    -o Global.pretrained_model=output/PPLCNetV2_base/best_model \
    -o Infer.infer_imgs=./images/0001.jpg

here -o Global.pretrained_model=“output/PPLCNetV2_base/best_model” Specifies the path where the current best weight is located , If you specify other weights , Just replace the corresponding path .

The default output is Top-5 Value , If you want to output Top-k Value , You can specify -o Infer.PostProcess.topk=k, among ,k The value specified for you .

6、 ... and 、 Based on the weights obtained from training inference Model

python tools/export_model.py \
    -c ppcls/configs/ImageNet/PPLCNetV2/PPLCNetV2_base.yaml \
    -o Global.pretrained_model=output/PPLCNetV2_base/best_model \
    -o Global.save_inference_dir=deploy/models/PPLCNetV2_base_infer

After the script is executed, it will be in deploy/models/ Lower generation PPLCNet_x1_0_infer Folder ,models The folder should have the following file structure ：

├── PPLCNet_x1_0_infer
│   ├── inference.pdiparams
│   ├── inference.pdiparams.info
│   └── inference.pdmodel

Use inference Model to predict .cd To deploy Directory ：

#  Use the following command to use  GPU  To make predictions 
python3 python/predict_cls.py -c configs/inference_cls.yaml -o Global.inference_model_dir=models/PPLCNet_x1_0_infer -o Infer.infer_imgs=./images/0001.jpg
#  Use the following command to use  CPU  To make predictions 
python3 python/predict_cls.py -c configs/inference_cls.yaml -o Global.inference_model_dir=models/PPLCNet_x1_0_infer -o Global.use_gpu=False -o Infer.infer_imgs=./images/0001.jpg

Folder based batch forecasting ：

#  Use the following command to use  GPU  To make predictions , If you want to use  CPU  forecast , You can add  -o Global.use_gpu=False
python3 python/predict_cls.py -c configs/inference_cls.yaml -o Global.inference_model_dir=models/PPLCNetV2_base_infer -o Global.infer_imgs=images/ImageNet/

7、 ... and 、paddle2onnx

paddle2onnx --model_dir ./  --model_filename inference.pdmodel --params_filename inference.pdiparams --save_file ./det_shv2.onnx --opset_version 11 --enable_onnx_checker True