Introduction and basic knowledge of machine learning

1. Introduction to machine learning

machine learning : Without programming directly for the problem , A research area that empowers computer learning

For a certain type of task T And performance metrics P, If a computer program T In order to P Performance measured with experience E And self-improvement , So we call this computer program from experience E Study

2. Common algorithms

2.1 Supervised algorithms

There are result marks in the sample data

classification

 According to the principle 

1. Based on Statistics Bayesian classification
2. rule-based Decision tree algorithm
3. Based on Neural Networks Neural network algorithm
4. Distance based KNN(K Nearest neighbor )

Common evaluation indicators

1. Accuracy Ratio of predicted results to actual results
2. Recall rate The correct coverage of certain results in the prediction results
3. F1-Score statistic , Comprehensive evaluation classification model Value 0-1 Between

Regression algorithm

2.1.1 KNN Algorithm

k-Nearest Neighbour One of the simplest classification algorithms , If the nearest to a sample k Most of the data in the samples belong to a certain category , Then it is considered that the sample also belongs to this category , And has the characteristics of the samples on this class ,KNN Can not only predict the classification , Regression analysis can also be done ( Predict specific values )

2.1.2 Decision tree algorithm

2.2. Unsupervised algorithm

There is no result mark in the sample data

2.2.1 clustering

1. Hierarchical clustering
2. Density clustering
3. Partition clustering

2.3 Semi supervised algorithm

Part of the sample data is marked with results

3. Detailed explanation of machine learning algorithm

3.1 Machine learning Basics

3.1.1 The basic concept of machine learning

1. input space : The set of all possible values of input is called input space
2. Output space : The set of all possible values of the output is called the output space
3. features : The property
4. Eigenvector : A vector composed of multiple features becomes a feature vector
5. The feature space : The space where the eigenvector exists is called the eigenspace
6. Hypothetical space : A set of mappings from input space to output space

3.1.2 The essence of machine learning

3.1.3 Three elements of machine learning methods

Method = Model + Strategy + Algorithm

1. Model : Mapping from input space to output space , Choose the appropriate assumption space
   1. 
2. Strategy : Learning criteria or rules for calculating rules from numerous hypothesis spaces to optimal models
   1. 
   2. Loss function : Used to measure the difference between the predicted results and the real results , The less it's worth , Represents the expected results and the real results It's usually a non negative real valued function , The process of reducing the loss function in various ways is called optimization , The loss function is recorded as L(Y,f(x))
      1. 0-1 Loss function If the predicted value is equal to the actual value, there will be no loss , Otherwise, it is a complete loss
      2. Absolute loss function : The absolute value of the difference between the predicted result and the real result
      3. Square loss function : The square of the difference between the predicted result and the real result
      4. Logarithmic loss function : Logarithmic functions are monotonic , When solving optimization problems , The result is consistent with the original goal , You can convert multiplication to addition
      5. Exponential loss function : monotonicity , Excellent properties of nonnegativity . Make the closer to the correct result, the smaller the error
      6. Folding loss function
   3. Empirical risk & Risk function
      1. 
   4. Structural risk
      1. 
      2. 

3.2 Model evaluation and selection

3.2.1 Principles of model selection

3.2.1.1 Basic concepts

1. error : The difference between the predicted output value of the model and its real value
2. Training : Learning through known sample data , The process of obtaining the model
3. Training error : The error between the model action and the training set
4. generalization : From special to general , For machine learning, it refers to applying new sample data from the model
5. The generalization error : The error of the new sample model
6. Model capacity : Ability to fit various models
7. Over fitting : A model performs well on the sample , Poor performance on new samples
8. Under fitting : The model does not perform well on the training set
9. Model selection : Choose the model with the least generalization error

3.2.2 Performance index of the model

3.2.3 The method of model evaluation

1. Set aside method : Use 80% The known data set is used as the training set to train the model , Use the rest of 20% Test the trained model as a test set , The test error obtained from the test set is used as the approximate value of the generalization error , Take the model with small test error
   1. Test set and training set shall be mutually exclusive as far as possible
   2. The test set and training set are independent and identically distributed
2. Cross validation : Divide the dataset into k Two mutually exclusive data subsets . Subset data is sampled hierarchically , Select one data set as the test set at a time , The rest are used as training sets , Conduct k Training and testing , Get the average , This verification method is called k Crossover verification   Use different divisions , repeat p Time , be called p Time k Crossover verification

3.2.4 Comparison of model performance

3.2.4.1 Performance measurement of regression algorithm

3.2.4.2 Performance measurement of classification algorithm