当前位置：网站首页>Explanation of CNN circular training | pytorch series (XXII)

Explanation of CNN circular training | pytorch series (XXII)

2022-07-28 02:52:00 【51CTO】

writing |AI_study

CNN Explanation of circular training | PyTorch series （ Twenty-two ）_ data

The original title ：CNN Training Loop Explained - Neural Network Code Project

Prepare the data
Build a model
Training models

Build training loop

Analysis of the results of the model

Single batch Training

We can put individual batch The training code is summarized as follows :

       
       network 
       
       = Network
       
       (
       
       )
       
train_loader = torch.utils.data
       
       .DataLoader
       
       (train_set, batch_size=100
       
       )
       
optimizer = optim
       
       .Adam
       
       (network
       
       .parameters
       
       (
       
       ), lr=0.01
       
       )
       
batch 
       
       = next
       
       (iter
       
       (train_loader
       
       )
       
       ) 
       
       # Get Batch
       
images, labels = batch
       
preds 
       
       = network
       
       (images
       
       ) 
       
       # Pass Batch
       
loss = F
       
       .cross_entropy
       
       (preds, labels
       
       ) 
       
       # Calculate Loss
       
loss
       
       .backward
       
       (
       
       ) 
       
       # Calculate Gradients
       
optimizer
       
       .step
       
       (
       
       ) 
       
       # Update Weights
       
       print
       
       (
       
       'loss1:', loss
       
       .item
       
       (
       
       )
       
       )
       
preds 
       
       = network
       
       (images
       
       )
       
loss = F
       
       .cross_entropy
       
       (preds, labels
       
       )
       
       print
       
       (
       
       'loss2:', loss
       
       .item
       
       (
       
       )
       
       )
      
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.

Output

       
       loss1: 
       
       2.3034827709198
       
       loss2: 
       
       2.2825052738189697
      
1.
2.

One thing you'll notice is that , Every time you run this code, you get different results . This is because the model is created at the top every time , We know from previous articles that the weight of the model is randomly initialized .

Now let's see how to modify this code to use all of the batch, So we can use the whole training set to train .

all batch Training for (epoch)

Now? , To train all the batches available in our data loader , We need to make some changes and add an extra line of code :

       
       network 
       
       = Network
       
       (
       
       )
       
train_loader = torch.utils.data
       
       .DataLoader
       
       (train_set, batch_size=100
       
       )
       
optimizer = optim
       
       .Adam
       
       (network
       
       .parameters
       
       (
       
       ), lr=0.01
       
       )
       
total_loss = 
       
       0
       
total_correct = 
       
       0
       
for batch in train_loader: 
       
       # Get Batch
       
    images, labels = batch
       
    preds 
       
       = network
       
       (images
       
       ) 
       
       # Pass Batch
       
    loss = F
       
       .cross_entropy
       
       (preds, labels
       
       ) 
       
       # Calculate Loss
       
    optimizer
       
       .zero_grad
       
       (
       
       )
       
    loss
       
       .backward
       
       (
       
       ) 
       
       # Calculate Gradients
       
    optimizer
       
       .step
       
       (
       
       ) 
       
       # Update Weights
       
    total_loss += loss
       
       .item
       
       (
       
       )
       
    total_correct +
       
       = get_num_correct
       
       (preds, labels
       
       )
       
       print
       
       (
       
       "epoch:", 
       
       0, 
       
       "total_correct:", total_correct, 
       
       "loss:", total_loss
       
       )
      
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.

We're going to create one for Loop to iterate all of batch Handle , Instead of getting a single batch Handle .

Because our training focus is on 60,000 Samples , So we're going to have 60,000 / 100 = 600 Sub iteration . For this reason , We're going to remove... From the loop print sentence , And track the total losses and the total number of correct predictions that were finally printed .

About this 600 One thing to note about the next iteration is , At the end of the loop , Our weights will be updated 600 Time . If we improve batch_size This number will go down if we lower batch_size This number will go up .

Last , Before we get to loss Tensor calls backward() After method , We know that gradients will be calculated and added to the network parameters grad Properties of the . on this account , We need to zero these gradients . We can use the zero_grad() Method to achieve this .

We're ready to run this code . This time the code will take longer , It's going to be because of the loop 600 Batches .

       
       epoch: 0 total_correct: 42104 loss: 476.6809593439102
      
1.

We got the result , We can see 60000 The correct total is 42104.

       
       > total_correct / len(train_set)
       
7017333333333333
      



There is only one epoch( A complete data transfer ) after , It's already very good . Even if we make one epoch, We still need to remember , Weights have been updated 600 Time , It depends on our batch size . If you allow batch_batch It's bigger , such as 10,000, Then the weight will only be updated 6 Time , It's not going to be very good either .

Multiple epoch Of Training

To execute multiple epoch, All we have to do is put this code into for In circulation . We will also put epoch Add a number to print In the sentence .

       
       network 
       
       = Network
       
       (
       
       )
       
train_loader = torch.utils.data
       
       .DataLoader
       
       (train_set, batch_size=100
       
       )
       
optimizer = optim
       
       .Adam
       
       (network
       
       .parameters
       
       (
       
       ), lr=0.01
       
       )
       
for epoch in 
       
       range
       
       (
       
       1
       
       0
       
       ):
       
    total_loss = 
       
       0
       
    total_correct = 
       
       0
       
    for batch in train_loader: 
       
       # Get Batch
       
        images, labels = batch
       
        preds 
       
       = network
       
       (images
       
       ) 
       
       # Pass Batch
       
        loss = F
       
       .cross_entropy
       
       (preds, labels
       
       ) 
       
       # Calculate Loss
       
        optimizer
       
       .zero_grad
       
       (
       
       )
       
        loss
       
       .backward
       
       (
       
       ) 
       
       # Calculate Gradients
       
        optimizer
       
       .step
       
       (
       
       ) 
       
       # Update Weights
       
        total_loss += loss
       
       .item
       
       (
       
       )
       
        total_correct +
       
       = get_num_correct
       
       (preds, labels
       
       )
       
       print
       
       (
       
       "epoch", epoch, 
       
       "total_correct:", total_correct, 
       
       "loss:", total_loss
       
       )
      
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.

After running this code , We get every one of them epoch Result :

       
       epoch 
       
       0 
       
       total_correct: 
       
       43301 
       
       loss: 
       
       447.59147948026657
       
       epoch 
       
       1 
       
       total_correct: 
       
       49565 
       
       loss: 
       
       284.43429669737816
       
       epoch 
       
       2 
       
       total_correct: 
       
       51063 
       
       loss: 
       
       244.08825492858887
       
       epoch 
       
       3 
       
       total_correct: 
       
       51955 
       
       loss: 
       
       220.5841210782528
       
       epoch 
       
       4 
       
       total_correct: 
       
       52551 
       
       loss: 
       
       204.73878084123135
       
       epoch 
       
       5 
       
       total_correct: 
       
       52914 
       
       loss: 
       
       193.1240530461073
       
       epoch 
       
       6 
       
       total_correct: 
       
       53195 
       
       loss: 
       
       184.50964668393135
       
       epoch 
       
       7 
       
       total_correct: 
       
       53445 
       
       loss: 
       
       177.78808392584324
       
       epoch 
       
       8 
       
       total_correct: 
       
       53629 
       
       loss: 
       
       171.81662507355213
       
       epoch 
       
       9 
       
       total_correct: 
       
       53819 
       
       loss: 
       
       166.2412590533495
      
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

We can see that the number of correct values has increased , and loss Less .

Complete training loop

Put all these together , We can put the Internet 、 Optimizer and train_loader Extracted from the training cycle unit .

       
       network 
       
       = Network
       
       (
       
       )
       
optimizer = optim
       
       .Adam
       
       (network
       
       .parameters
       
       (
       
       ), lr=0.01
       
       )
       
train_loader = torch.utils.data
       
       .DataLoader
       
       (
       
    train_set
       
    ,batch_size=100
       
    ,shuffle=True
       
       )
      
1.
2.
3.
4.
5.
6.
7.

optimizer = optim.Adam(network.parameters(), lr=0.01)

       
       for epoch 
       
       in range(10):
       
    total_loss 
       
       = 
       
       0
       
    total_correct 
       
       = 
       
       0
       
       for batch 
       
       in train_loader: 
       
       # Get Batch
       
        images, labels 
       
       = batch
       
        preds 
       
       = network(images) 
       
       # Pass Batch
       
        loss 
       
       = F.cross_entropy(preds, labels) 
       
       # Calculate Loss
       
        optimizer.zero_grad()
       
        loss.backward() 
       
       # Calculate Gradients
       
        optimizer.step() 
       
       # Update Weights
       
        total_loss 
       
       +
       
       = loss.item()
       
        total_correct 
       
       +
       
       = get_num_correct(preds, labels)
       
    print(
       
       "epoch", epoch, 
       
       "total_correct:", total_correct, 
       
       "loss:", total_loss
       
    )
      
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.

And then there's the visualization

We should now have a good understanding of the training cycle and how to use it PyTorch To build them .PyTorch The cool thing is , We can be like debugging forward() Function to debug the training loop code like that .

In the next article , We'll see how to get predictions for each sample in the training set , And use these predictions to create a confusion matrix . See you next class !

The content of the article is carefully studied , My level is limited , Translation cannot be perfect , But it really took a lot of effort , I hope you can move your sexy hands , Share a circle of friends , Support me ^_^

The original English link is ：

https://deeplizard.com/learn/video/XfYmia3q2Ow

CNN Explanation of circular training | PyTorch series （ Twenty-two ）_ data _02