Deep learning --- the weight of the three good students' scores (3)

 

🥰 Blog's front page ：knighthood2001

Comments are welcome on thumb up ️

️ love python, Looking forward to progress and growth with you ！！️

         Before , Let's start with the three good students' grades , A neural network model is designed to solve this problem , But this model is more designed from the general way of thinking , This is not consistent with the idea of neural network design , So this article , Let's see how to optimize the model , Make its logic clearer 、 Run more efficiently .

Catalog

All the code

Code explanation  

Result display  

---

All the code

import tensorflow as tf

tf.compat.v1.disable_eager_execution()

x = tf.compat.v1.placeholder(shape=[3], dtype=tf.float32)
yTrain = tf.compat.v1.placeholder(shape=[], dtype=tf.float32)

w = tf.Variable(tf.zeros([3]), dtype=tf.float32)

n = x * w

y = tf.reduce_sum(n)

loss = tf.abs(y - yTrain)

optimizer = tf.compat.v1.train.RMSPropOptimizer(0.001)

train = optimizer.minimize(loss)

sess = tf.compat.v1.Session()

init = tf.compat.v1.global_variables_initializer()

sess.run(init)

for i in range(5000):
    result = sess.run([train, x, w, y, yTrain, loss], feed_dict={x: [90, 80, 70], yTrain: 85})
    print(result)

    result = sess.run([train, x, w, y, yTrain, loss], feed_dict={x:[98, 95, 87], yTrain: 96})
    print(result)

Code explanation  

         In the neural network model designed before , The moral education of students 、 Intellectual education 、 sports 3 The item scores correspond to x1、x2、x3 This is an input layer node , There is no problem in this way , But suppose you add another art score ,  Then you need to add a in the input layer x4 node , In the hidden layer, you should also add a n4 node . in other words , When the input data changes , Even if the whole logic doesn't change , We also need to modify the whole network model , More trouble .

         in addition , A large number of nodes will also make the model diagram look more complex . So in the usual neural network , Many times, the data of this string will be organized into a  “ vector ” To feed into the neural network for calculation . there “ vector ” It is slightly different from the concept of vector in mathematical geometry , It means a string of numbers , Use an array in the program to represent , for example , The first student in the "three good" student achievement problem 3 The item score can be expressed as [90, 80, 70] Such an array is represented . Arrays are ordered , It can be agreed that the first item represents moral education 、  The second item represents the score of intellectual education 、  The third item represents sports score . There are several numbers in the vector , Generally, it is called several “ dimension ” Vector , For example, this vector is a three-dimensional vector .

x1 = tf.compat.v1.placeholder(dtype=tf.float32)
x2 = tf.compat.v1.placeholder(dtype=tf.float32)
x3 = tf.compat.v1.placeholder(dtype=tf.float32)

Instead, a three-dimensional vector is stored in the variable x

x = tf.compat.v1.placeholder(shape=[3], dtype=tf.float32)

  Variable x The definition statement of is slightly different , A named parameter is added shape, This is the representation variable x Morphological , It's going to be theta “[3]”,  Represents an input placeholder x The data will be one with 3 Array of numbers , That is, a three-dimensional vector . 

w1、w2、w3 this 3 Variable parameters are also reduced to a three-dimensional vector w:

w = tf.Variable(tf.zeros([3]), dtype=tf.float32)

among tf.Variable The first parameter of the function is the initial value of the variable parameter , because x Is a form of [3] Three dimensional vector ,w It also needs to be a form corresponding to [3] Three dimensional vector , And we use tf.zeros This function can generate a value of all 0 Vector , in other words tf.zeros([3]) The return value of will be an array [0, 0, 0], This vector will be used as w Of Initial value . 

yTrain Because it's just an ordinary number , It's not a vector , If you want to give it a form ,  You can use an empty square bracket “[]” To represent the . 

yTrain = tf.compat.v1.placeholder(shape=[], dtype=tf.float32)

Hidden layer node variables n1、n2、n3 Also reduced to a variable  n: 

n = x * w

Hypothetical input data x by [90, 80, 70], That is, the first student 3 Item score , here w by [2,3,4], that n=x*W The result of this operation is [90 * 2, 80* 3, 70* 4], namely [180, 240,280]. This is because “*” Representing matrix operations in mathematics “ Point multiplication ”, Point multiplication refers to the multiplication of numbers at each same position in two matrices with the same form , The result is a matrix with the same form as the two matrices . The method of vector point multiplication is the same as that of matrix point multiplication ,  therefore x * w The result of the calculation is the same as x or w A three-dimensional vector of the same shape , The result of the first dimension is  x The number of the first dimension in  90 multiply w The number of the first dimension in 2,  namely 90*2 obtain 180, And so on

Because we put the original 3 A hidden layer node n、n2、n3 Reduced to a vector  n, Output layer node y The calculation of the has to be changed : 

y = tf.reduce_sum(n)

tf.reduce sum The function takes the vector as its parameter ( In the future, it may be a matrix ) The values of all dimensions in are added and summed , With the original y=n1 +n2 + n3 The meaning is the same .

    result = sess.run([train, x, w, y, yTrain, loss], feed_dict={x: [90, 80, 70], yTrain: 85})
    print(result)

    result = sess.run([train, x, w, y, yTrain, loss], feed_dict={x:[98, 95, 87], yTrain: 96})
    print(result)

Finally, just change here  

Result display  

We finally repeat the cycle 5000 Time

Look at the last few results , We can see , The error is also controlled to a small extent ,3 The weights are also close to the expected values .