当前位置：网站首页>Learning notes of statistical learning methods -- Chapter 1 Introduction to statistical learning methods

Learning notes of statistical learning methods -- Chapter 1 Introduction to statistical learning methods

2022-07-05 21:25:00 【Raymond。】

Statistical learning methods learning notes -- Chapter one Introduction to statistical learning methods

1.1 Statistical learning
1.2 Supervised learning

1.1 Statistical learning

1.1.1 Basic steps of statistical learning

Steps to achieve statistical learning methods ：

Get a limited set of training data
Determine the hypothetical space containing all possible models , That is, the set of learning models
Determine the criteria for model selection , Learning strategies
Determine the algorithm for solving the optimal model , Learning algorithm
Choosing the best model by learning method
Use the learned optimal model to predict and analyze new data

1.1.2 Statistical learning classification

Statistical learning includes supervised learning , Unsupervised learning , Semi supervised learning and intensive learning . Focus on supervised learning .

1.1.3 Three elements of statistical learning method

Statistical learning method = Model + Strategy + Algorithm

Model
A set of all possible mappings from input variables to output variables
Strategy
The optimal model .
How to measure the quality of the model ： Loss function L(Y, f(X))( Measure the quality of a forecast ) And risk function ( The prediction of the model is good or bad in the average sense ).
Algorithm
The calculation method of solving the optimal model .

1.1.4 Model evaluation and model selection

Training error and test error
Over fitting
The training error is small , The test error is large .( The noise is also studied and predicted )

1.1.5 Regularization and cross validation -- Prevent over fitting

Regularization
Add regularization term or penalty term to empirical risk , Measure the complexity of the model .
Cross validation
When the data is enough , Divide the data into training sets , Verification set ( For model selection ) And test set .

1.1.6 Generalization ability

1.1.7 Generation model and discrimination model

Generate models
Joint distribution by data learning P(X,Y), Find the conditional probability distribution P(Y|X).
Common generation models ： Naive Bayes and hidden Markov model
Discriminant model
Learning decision function directly from data f(X) Or conditional probability distribution P(Y|X).
Common discriminant models ：k a near neighbor , perceptron , Decision tree , Logistic regression model , Maximum entropy model , Support vector machine , Lifting method and condition random field .

1.2 Supervised learning

1.2.1 Basic concepts

input space , Feature space and output space
Input ( Output ) Space is input ( Output ) All possible values . Each specific input is an instance , Usually represented by eigenvectors , The space where all eigenvectors exist is the eigenspace .
Classification of prediction problems
The problem that input and output are continuous variables is called regression problem .
The output variables are finite discrete variables, which is called classification problem .
The prediction problem in which both input and output variables are variable sequences is called marking problem .
Hypothetical space
The purpose of supervised learning is to learn a mapping from input to output , A map is represented by a model , The set of all mappings is called the hypothesis space .
Supervised learning model classification
It can be divided into probability models ( By conditional probability distribution P(Y|X) Express ) And non probabilistic models ( Decision function Y=f(X) Express ). The specific model is determined by the specific learning method .