12. Optimization problem practice

himmelblau function ：

This function is used to detect the effect of the optimizer . You can solve this function , Take a look at the optimizer we designed , Or the gradient descent rule we designed , Can we find this solution well , Our solution is known . The difference between the two can be compared .
The minimum value is known ：
f(3.0,2.0)=0
f(-2.805118,3.131312)=0
f(-3.779310,-3.283186)=0
f(3.584428,-1.848126)=0
Are global minimum values .

import torch
from matplotlib import pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D


def himmelblau(x):
    return (x[0]**2+x[1]-11)**2+(x[0]+x[1]**2-7)**2

#  Visual code , Print out the function image 
# Generate x and y Axis data list 
x = np.arange(-6,6,0.1)
y = np.arange(-6,6,0.1)
print('x,y range:',x.shape,y.shape)
# Yes x and y Grid the data 
X,Y = np.meshgrid(x,y)
print('X,Y maps:',X.shape,Y.shape)
Z = himmelblau([X,Y])

fig=plt.figure('himmelblau')
ax =fig.add_subplot(projection='3d')
ax.plot_surface(X,Y,Z)
ax.view_init(60,-30)
ax.set_xlabel('x')
ax.set_xlabel('y')
plt.show()

#x Can be initialized to [1.,0.],[-4,0.],[4,0.]
x=torch.tensor([0.,0.],requires_grad=True)
optimizer = torch.optim.Adam([x],lr=1e-3)
for step in range(20000):
    pred = himmelblau(x)

    optimizer.zero_grad()
    pred.backward()
    optimizer.step()

    if step % 2000 == 0:
        print('step {}: x={},f(x)={}'.format(step,x.tolist(),pred.item()))

Transformation x The initial value of the , You can get the other three minima .
optimizer = torch.optim.Adam([x],lr=1e-3)
The goal of the optimizer is x, This step will be completed automatically ：
x‘=x-0.001▽x
y‘=y-0.001▽y
Just do it once optimizer.step(), The above two processes will be updated once .
optimizer.zero_grad() Clear the gradient information .
pred.backward() Generate x and y Gradient information ▽x and ▽y.
Non stop cycle , Until you find the right x and y.

modify x by ：x=torch.tensor([-4,0.],requires_grad=True)