Paper reading notes -- crop yield prediction using deep neural networks

Khaki S, Wang L. Crop yield prediction using deep neural networks[J]. Frontiers in plant science, 2019, 10: 621.

1、 Data processing

Estimate missing values to preprocess data , Define a threshold as the maximum number of missing values allowed for each genetic marker . If the missing value of genetic marker exceeds the defined threshold , Throw it away .

In data mining , It is generally recommended not to set too large a threshold , Because it will increase the error and noise in the data set , This leads to very poor predictions .

After deleting genetic markers with too many missing values according to the threshold , All missing values will be estimated . In statistics , Interpolation is the process of replacing missing data with substitute values , Instead of deleting instances with missing values . Imputation preserves observations by replacing missing data with estimates based on other available information . Once all missing values have been estimated , Standard techniques can be used to analyze data sets to obtain complete data . There are many methods for missing value interpolation , For example, the mean value 、 The median 、 Most frequent and random regression methods .

Because the genetic data set contains only three values （-1、0、1） Classified data , So we tried the median and the most common method , Among them, the median method proves to be better in the verification set . therefore , Use the median method for interpolation .

2、 Method

Neural networks usually have many weights and deviations to train . The training process is cyclical , The following steps are included ：

• Forward propagation ： At this stage , The input data is input into the network , And spread forward layer by layer through the network , Until you reach the output layer .

• Calculate the loss ： At this stage , Compare the output with the actual target value , To see the accuracy of the current model .

• Back propagation ： At this stage , Use the chain rule in calculus , Calculate the gradient of the loss function for all weights and deviations of all layers in the network .

• to update ： At this stage , We use the gradient based optimization method to update all weights and deviations , For example, random gradient descent (SGD)、Adam、Adagrad optimization algorithm .

Deeper networks are harder to train , Or they have gradient vanishing problem . By back propagation , The gradient flows back from the output layer to the beginning layer of the network , This includes many multiplication of usually smaller values . therefore , When it reaches the first layer of neural network , Sometimes the gradient will be almost zero . therefore , The weight of the first layer will not be updated , This leads to high errors .

To solve this problem , Normalize the input data , This helps to use random gradient descent (SGD) To help deeper network convergence . The gradient degradation problem is alleviated by introducing a deep residual learning framework . They do not fit expressed as H(x) The required underlying mapping , Instead, let the stacked nonlinear layers fit F(x) Residual mapping .

They assume that it is easier to optimize the residual mapping than to optimize the original network . In the residual network , Every two stack layers have a quick connection . Quick connect is just an add door , In back propagation , Adding doors preserves gradients , And act as a gradient highway in the whole network . therefore , We ensure that all weights in the network will be fully updated through the cycle of forward and back propagation .

They do not add additional parameters to the network , It doesn't increase the computational complexity . The whole network is still accessible through SGD Use back propagation for end-to-end training , And you can use Tensorflow and Pytorch And so on , Without modifying the solver .

In this paper , They trained a depth 21 Layer of neural network , Used for production and checking production forecasts . We used the residual shortcut after every two stacking layers of the network . Batch normalization is also used for each middle layer to obtain better convergence . Last , Use Adam Optimizer optimizes network . The whole process uses Tensorflow Open source software library in Python 3.5 Code in .

The following implementation is used ：

• 21 Hidden layers

• Each layer 40 Neurons

• Learning rate 0.0003 Of Adam Optimizer

• The exponential decay rate of learning rate is 0.95

• Batch size is 64

• Batch standardization of all hidden layers

• 300,000 Sub iteration

• The remaining shortcuts

• 143,999 Training samples

• 4453 Samples as validation set （ Training set 0.03）

• Relu Family activation function

• Of all weights Xavier initialization

Verification method ：

Regression evaluation index MSE、RMSE、MAE、R-Squared_sljwy The blog of -CSDN Blog

3. Weather forecast neural network

Need to predict 2017 Annual weather variables . There is 315 A different position , Each location should be predicted 72 Weather variables .

Treat each variable at any location as a single time series . For time series analysis , A hidden layer neural network has been trained . however , First, you should create training data . Weather data is usually cyclical , Therefore, the value of a specific variable next year may only depend on certain quantities in previous years . Due to crop challenge, information about variable names is not provided in the weather dataset , So all 72 All weather variables are considered to be between 1 To 13 All correlations between years （ lagging ）, And the least mean square error (MSE) the .

For each variable , Yes 315 A place , All positions of a specific variable together create a training data set according to the above method . Last , With 2001 Year to 2015 The weather data of the year is used as the training set ,2016 The weather data of the year is used as the verification set . All during validation 72 Average of variables MSE by 0.18. then , The results of this section are used in the production forecasting process .