Deep learning | three concepts: epoch, batch, iteration

• Epoch（ period ）：
  When a complete data set passes through the neural network once and returns once , This process is called a time >epoch.（ in other words , All training samples In the neural network all the A forward propagation and A back propagation ）
  A little more general , One Epoch Namely Train all training samples once The process of .

However , When one Epoch The sample of （ That's all the training samples ） The number may be too large （ For computers ）, You need to break it into small pieces , That is, share Multiple Batch To train .**

• Batch（ batch / A batch of samples ）：
  Divide the whole training sample into several Batch.

• Batch_Size（ Batch size ）：
  The size of each batch of samples .

• Iteration（ One iteration ）：
  Train one Batch Just once Iteration（ This concept is similar to iterators in programming languages ）.

• Why use more than one epoch?

It is not enough to pass a complete data set in a neural network at one time , And we need to pass the complete data set many times in the same neural network . But remember , We use a limited set of data , And we use an iterative process called gradient descent to optimize the learning process . As shown in the figure below . So just update it once or use one epoch It's not enough. .

With epoch increase in numbers , The number of updates of weights in neural networks is also increasing , The curve changes from under fitting to over fitting .

that , The problem is coming. , How many? epoch That's the right thing to do ？

Unfortunately , There is no right answer to this question . For different data sets , The answer is different . But the diversity of data can affect the right epoch The number of . such as , Only the black cat dataset , And data sets of cats in all colors .

Conversion relation ：

actually , gradient descent The fundamental difference between the above methods lies in the  Batch_Size  Different .

for instance ：