Machine Learning - Gradient Descent Optimization - C language implementation

1.Introduction

Gradient descent is a method in optimization theory to find the best solution. The purpose is to use gradient descent to find the local minimum of a function.
This article will show you how to implement a gradient descent solver yourself based on the basic principles of gradient descent, focusing on how to do it.

2. Gradient Descent of Unary Functions

First of all, starting from a simple one, only considering the unary function, how do we implement the gradient descent method?
The optimization function is: f(x) = x^2-10x+1
Steps of gradient descent:
x(t+1) = x(t) - Y*df(x(t))/dx(t),
Calculate x(0),x(1),x(2),x(3),..., we can solve f(0), f(1), f(2), f(3), ... through f(x) = x^2-10x+1, when f(k+1)-f(k) is very small, and we think that x(k+1) is the solution we are looking for.
So for the optimization function: f(x) = x^2-10x+1, the iterative formula is:
x(k+1) = x(k) - Y*(2*x(k)-10), where Y is the learning rate, generally a number between 0-1.

The c language code for the implementation of this example is:

/* * Description:Gradient Descent Optimization * x(t+1) = x(t) - Y*df(x(t))/dx *f(x) = x^2-10x+1 */