Machine Learning (2) - Machine Learning Fundamentals

You can follow share16 on Zhihu or WeChat public account, we will also update this article synchronously.

I. Supplementary knowledge

1.1 xgboost

   xgboost and LightGBM are the most commonly used prediction algorithms in data mining.xgboost is an engineering implementation of the GBDT (gradient boosting tree) algorithm. Its logic is: first train a tree, then calculate the error (also called gradient) of each sample, train the next tree by fitting the error, and keep fittinguntil the training results meet the requirements.

import xgboost as xgb# from xgboost.sklearn import XGBClassifierxgb.XGBClassifier()# objective="binary:logistic" The objective function, representing the binary classification, the loss function is logistic

1.2 LightGBM

   LightGBM can be said to be a continuation of xgboost. Some optimizations have been made on xgboost. From the application level, the two most direct points are: ① The speed is much faster; ② It can support category-type features (in xgb it isNo, one-hot is required).

import lightgbm as gbmgbm.LGBMClassifier()

1.3 BP Neural Network

   BP neural network is a multi-layer feedforward neural network trained according to the error back-propagation algorithm. It is one of the most widely used neural network models and the theoretical basis of current deep learning.The current parameter optimization of deep learning network is based on BP and gradient regression.The BP neural network is divided into a forward network and a backward network. The forward network is used to calculate the target value, and the backward network is used for gradient calculation.

from sklearn.neural_network import MLPClassifierBP = MLPClassifier()

Thank you all