Tensorflow customize the whole training process

Create a machine learning problem

$$f(x)=3x+7$$
For a machine learning problem , There are the following steps ：

1. Get training data .
2. Defining models .
3. Define the loss function .
4. Traverse the training data , Calculate the loss from the target value .
5. Calculate the gradient of this loss , And use optimizer Adjust variables to fit data .
6. The result of the calculation is .

Build data

Supervised learning uses input （ Usually expressed as x） And the output （ Expressed as y, Commonly known as labels ）. The goal is to learn from paired inputs and outputs , So that you can predict the output value according to the input .TensorFlow Almost every input data in is represented by tensor , And it's usually a vector . In supervised learning , Output ( That is, think of the predicted value ) It's also a tensor . This is done by putting Gauss （ Normal distribution ） Some data synthesized by adding noise to the points on the line , And visualize these data .

x = np.random.random([1000]) * 5
noise = np.random.random([1000])
y = 3 * x + 7
import matplotlib.pyplot as plt
plt.scatter(x, y, c="b")
plt.show()

Customize the model we need

We inherit tf.module class , And define two variables , Its attribute is trainable_variables.

class selfmodel(tf.Module):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        self.v1 = tf.Variable(1.0, trainable=True)
        self.v2 = tf.Variable(2.0, trainable=True)
    def __call__(self, x):
        y = self.v1 * x  + self.v2
        return y

Define the loss function

We use the mean square deviation here to calculate the loss

def loss(target_y, predicted_y):
    return tf.reduce_mean(tf.square(target_y - predicted_y))

Define the cycle training function

We use epcohs Function to get the two variables we need $v1,v2$, After each training $v1,v2$ Record , Finally, visualization

def train(model, x, y,epochs,optimizer):
    v1, v2 = [], []
    for j in range(epochs):
        with tf.GradientTape() as gd:
            y_pred = model(x)  # This needs to be inside 
            loss_score = loss(y, y_pred)
        grad = gd.gradient(loss_score, model.trainable_variables) 
        optimizer.apply_gradients(zip(grad, model.trainable_variables))
        v1.append(model.v1.numpy())
        v2.append(model.v2.numpy())
    return (model, v1, v2)

The end result shows

I'm defining epcohs When , If the setting is too small , Will lead to $v1,v2$ Can't get the right result

opt = tf.keras.optimizers.SGD()
model = selfmodel()
epochs = 1000
(model, v1, v2) = train(model, x, y,epochs, opt)
#  draw 
plt.plot(range(epochs), v1, "r",
         range(epochs), v2, "b")
plt.plot([3] * epochs, "r--",
         [7] * epochs, "b--")
plt.legend(["W", "b", "True W", "True b"])
plt.show()

Problems in the code

# The code in this case will report an error , Say our grad The result is （none, none）,
# because y_pred = model(x) It should be written in with Inside 
# The following will write the correct way , The reason for this error is loss Function in 
# Yes molel.trainable_variables When seeking derivative , Gradient not found 
y_pred = model(x)
with tf.GradientTape() as t:
    l = loss(y, y_pred)
grad = t.gradient(l, model.trainable_variables)
optimizer = tf.keras.optimizers.SGD()
optimizer.apply_gradients(zip(grad, model.trainable_variables))

# Correct writing 
with tf.GradientTape() as t:
	y_pred = model(x)
    l = loss(y, model(x))
grad = t.gradient(l, model.trainable_variables)
print(model.trainable_variables)
optimizer = tf.keras.optimizers.SGD()
optimizer.apply_gradients(zip(grad, model.trainable_variables))