Basic concepts of machine learning

Data analysis is the essence of most natural sciences , Extract rules from daily observations , And look for uncertainty .

Storage capacity fails to keep pace with the growth of data volume . meanwhile , The growth of computing power has overshadowed the growth of data volume .

Capacity control method , Such as discarding

Attention mechanism ： Expand the memory capacity and complexity of a system without increasing parameters .

Machine learning studies how to make computer systems use experience to improve performance

features （feature）： Used to characterize the sample , Used to predict labels （label） The two factors of are called characteristics .

Characteristic number / Eigenvector dimension ： Number of input layers

Analytic solution （analytical solution）： When the model and loss function are simple , The solution of the error minimization problem can be directly expressed by formula .

Numerical solution （numerical solution）： Most deep learning models have no analytical solution , The value of the loss function can only be reduced as much as possible by optimizing the model parameters for a finite number of iterations of the algorithm .

Small batch random gradient drop （mini-batch stochastic gradient descent）： Numerical solution of the optimization algorithm is the most widely used

 ∣B∣  Represents the number of samples in each small batch （ Batch size ,batch size）,η Learning rate （learning rate） And take a positive number

Hyperparameters （hyperparameter）： namely “ Adjustable parameter ” Parameters in , Artificially set , Not through model training .

Fully connected layer （fully-connected layer） Or dense layer （dense layer）： The neurons in the output layer are completely connected to each input in the input layer

Optimization 

 Neuron Network 

Optimization Fails

 

eigen vector Eigenvector

Adaptive Learning Rate

Root Mean Square

 RMSProp

Loss

likelihood （likelihood）

Batch Normalization

x feature vector

R Training data set

mean Average

deviation deviation

dim dimension

standardization Standardization ∈normalization

converge Converge

⊙  

At the beginning of training ,β by 0 vector ,γ by 1 vector

ps： If downloading package is slow , can   Add domestic image source

give an example ：

pip install To install  https://pypi.tuna.tsinghua.edu.cn/simple opencv-contrib-python

pip install opencv-python https://pypi.tuna.tsinghua.edu.cn/simple opencv-contrib-python

Or forever Set up pip The download source is domestic .

pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

Reference material ：

( Strong push ) Li Hongyi 2021 Spring machine learning course _ Bili, Bili _bilibili

Dive-into-DL-PyTorch