2021 Li Hongyi machine learning (1): basic concepts

B On the site 2021 Li Hongyi's learning notes of machine learning course , For reuse .

1 Basic concepts

Machine learning is ultimately about finding a function .

1.1 Different function categories

• Return to Regression—— Output is numeric
• classification Classification—— The output is in different categories classes, Do multiple choice questions
• Structural learning Structured Learning—— Generate a structured file （ Draw a picture 、 Write an article ）, Let the machine learn to create

1.2 How to find functions （Training）：

• First , Write a function with unknown parameters ;
• secondly , Definition loss（ A function related to parameters ,MAE—— Absolute error ,MSE—— Mean square error ）;
• Last , Optimize , Find the loss Minimum parameters —— gradient descent
  １） Randomly select the initial value of the parameter ;
  ２） Calculation ${\frac{\partial L}{\partial w}|}_{w=w^0}$, Then step down the gradient , The step size is 　${lr\times \frac{\partial L}{\partial w}|}_{w=w^0}$
  ３） Update parameters
  This method has a huge drawback ： Usually we will find Local minima, But what we want is global minima

1.3 Model

Linear model linear model There's a big limit , Cannot simulate polyline 、 Curve , This restriction is called model bias, So we need to improve .
How to improve ：Piecewise Linear Curves

Many such sets can be fitted into curves .

1.3.1 sigmoid

It can be used sigmoid function $y=c\frac{1}{1+e^{-\left(b+w{x}_1\right)}}＝ｃsigmoid(b+w{x}_1)$ Fit the blue broken line ：
$$y=b+\sum _i{c}_i\mathrm{sigmoid}\left(b_i+\sum _j{w}_{ij}{x}_j\right)$$

All unknown parameters in this , Use both $\theta$ Express ：

Use all at once $\theta$ To calculate , Make a gradient descent , Such a large amount of data , Therefore, small batches are used batch：

every last data The number of updates depends on the total amount of data and batch Number ：

1.3.2 ReLU

In front of it is soft sigmoid, That's the curve , In fact, you can use two ReLU Quasi synthesis hard sigmoid, That's the broken line ：

above sigmoid The formula becomes ：

1.3.3 Yes sigmoid The calculation of can be done several more times

There are many such layers , It is called neural network Neural Network, Later called Deep learning=Many hidden layers