DL: deep learning model optimization model training skills summary timely automatic adjustment of learning rate implementation code

DL： Deep learning model optimization model training skills summary of timely automatic adjustment of learning rate implementation code

Catalog

Deep learning model optimization model training skills summary timely automatic adjustment of learning rate

Implementation code

---

Deep learning model optimization model training skills summary timely automatic adjustment of learning rate

Implementation code

def scheduler(epoch):
    #  every other 50 individual epoch, Reduced learning rate to original 1/10
    if epoch % 50 == 0 and epoch != 0:
        lr = K.get_value(GRU.optimizer.lr)
        if lr>1e-5:
            K.set_value(GRU.optimizer.lr, lr * 0.1)
            print("lr changed to {}".format(lr * 0.1))
    return K.get_value(GRU.optimizer.lr)

reduce_lr = LearningRateScheduler(scheduler)
early_stopping = EarlyStopping(monitor='loss', 
                               patience=20, 
                               min_delta=1e-5,
                               mode='auto',
#                                restore_best_weights=False,# Whether to restore the model weight from the period with the best value of the monitored quantity 
                               verbose=2)