当前位置：网站首页>Realize batch data enhancement | use of keras imagedatagenerator

Realize batch data enhancement | use of keras imagedatagenerator

2022-07-28 09:05:00 【51CTO】

WeChat official account ：AI Algorithm and image processing

Content catalog

What is data imbalance? Here is a brief explanation , Focus on operation and Application ：1、 Data imbalance 2、 Common solutions function description preparation function description parameter description parameter description complete code effect display summary

What is data imbalance

Here is a brief explanation , Focus on operation and Application ：

1、 Data imbalance

In most cases , We believe that different categories of data are evenly distributed , Many algorithms are also based on this assumption , But in the real world , It is often not so . for example , The situation of machine sending failure is what we want to predict , But in fact, the probability of failure is very low , Therefore, the sample size leading to failure is very small , Even if you set all the prediction results to normal , The accuracy is still high , But this model is useless , Such similar examples are very common .

2、 Common solutions

There are many solutions , Mainly from two aspects （ During the interview, you may ask ）

In the project , We may not have so much time to think about solving it from the aspect of Algorithm , Most of the time, I think that I can use it , But many online examples are based on built-in data , This is very uncomfortable , Or data enhancement based on a picture , A lot of pain . More generally , Data enhancement for all pictures in a folder under the training set , That's why I wrote this .

Function description

preparation

The environment used

Functional specifications

1）ImageDataGenerator Image generator

Parameter description

remarks ： Data enhancement for a single image , You can see ImageDataGenerator and flow Instructions for use

       
       from 
       
       keras.
       
       preprocessing.
       
       image 
       
       import 
       
       ImageDataGenerator
       
       datagen 
       
       = 
       
       ImageDataGenerator(
       
       rotation_range
       
       =
       
       40,
       
       width_shift_range
       
       =
       
       0.2,
       
       height_shift_range
       
       =
       
       0.2,
       
       rescale
       
       =
       
       1.
       
       /
       
       255,
       
       shear_range
       
       =
       
       0.2,
       
       zoom_range
       
       =
       
       0.2,
       
       horizontal_flip
       
       =
       
       True,
       
       fill_mode
       
       =
       
       'nearest'
       
       ,
       
       cval
       
       =
       
       0,
       
       channel_shift_range
       
       =
       
       0,
       
       horizontal_flip
       
       =
       
       False,
       
       vertical_flip
       
       =
       
       False,
       
       rescale
       
       =
       
       None)
      
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.

Parameters

rotation_range： Integers , The random rotation angle of the picture when the data is raised

2）flow_from_directory

       
       gen 
       
       = 
       
       datagen.
       
       flow_from_directory(
       
       path,
       
       target_size
       
       =(
       
       224, 
       
       224),
       
       batch_size
       
       =
       
       15,
       
       save_to_dir
       
       =
       
       dst_path,
       
       # The saved path of the generated image 
       
       save_prefix
       
       =
       
       'xx',
       
       save_format
       
       =
       
       'jpg')
       
       for 
       
       i 
       
       in 
       
       range(
       
       6):
       
       gen.
       
       next()
       
       ""
       
       "
       
       path:
       
        File read path , Must be one level above the subfolder （ This is a pit , But try it and you'll understand ）
       
       target_size:
       
        picture resize Size of assembly , If it is not set, it will be set to （256
       
       .256
       
       ）
       
       batch_size:
       
        The number of pictures entered each time , for example batch_size
       
       =
       
       32
       
       , The number of enhancements at one time is 32,
       
        Personal experience ：batch_size The size of should be divisible with the number of files 
       
       save_to_dir:
       
        Where to save the enhanced picture 
       
       save_prefix:
       
        Prefix the file name , Convenient view 
       
       save_format:
       
        Save the data format of the picture 
       
        Total number of pictures produced ：batch_size
       
       *
       
       6
       
       （ namely range Number in ）
       
       ""
       
       "
      
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.

Complete code

       
       from 
       
       keras.
       
       preprocessing.
       
       image 
       
       import 
       
       ImageDataGenerator
       
       path 
       
       = 
       
       'E:/C3D_Data/train' # 
       
        One level above the category subfolder 
       
       dst_path 
       
       = 
       
       'E:/C3D_Data/train_result'
       
# 　
       
        Image generator 
       
       datagen 
       
       = 
       
       ImageDataGenerator(
       
       rotation_range
       
       =
       
       5,
       
       width_shift_range
       
       =
       
       0.02,
       
       height_shift_range
       
       =
       
       0.02,
       
       shear_range
       
       =
       
       0.02,
       
       horizontal_flip
       
       =
       
       True,
       
       vertical_flip
       
       =
       
       True
       
)
       
       gen 
       
       = 
       
       datagen.
       
       flow_from_directory(
       
       path,
       
       target_size
       
       =(
       
       224, 
       
       224),
       
       batch_size
       
       =
       
       15,
       
       save_to_dir
       
       =
       
       dst_path,
       
       # The saved path of the generated image 
       
       save_prefix
       
       =
       
       'xx',
       
       save_format
       
       =
       
       'jpg')
       
       for 
       
       i 
       
       in 
       
       range(
       
       6):
       
       gen.
       
       next()
      
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.

Effect display

Realize batch data enhancement | keras ImageDataGenerator Use _ Parameter description

Realize batch data enhancement | keras ImageDataGenerator Use _ data _02

summary

1、 It can basically meet the requirements of data enhancement we need

There is a point to emphasize here ,ImageDataGenerator It should be that every picture will be randomly operated （ For example, cutting , Spin, etc ） The role of

I hope every carefully written tweet can help you , Maybe you are already a great God , These are of no use to you , But this is a problem that novices will encounter at the beginning , I don't want you to give up because of a little setback , So tweeting on the one hand is to record the pit you really encounter , On the other hand, I also hope to improve my expression ability .
The code has been updated to ：https://github.com/SCUTPZW/AI_study/tree/master
I hope I can get your sincere star（ It hasn't been clicked yet ）

Realize batch data enhancement | keras ImageDataGenerator Use _ resolvent _03