【1】 Integrated learning: quickly understand Integrated Learning

One 、 The concept of ensemble learning

Integrated learning (ensemble learning) By building and combining Multiple learners To get the job done , Sometimes called Multi classifier system （multi-classifer system）————《 machine learning 》 zhou

( Learners mostly refer to what we call classifiers , For example, support vector machine （SVM）、 Decision tree 、 be based on BP Neural network of algorithm, etc )
below , Let's briefly talk about my understanding , If you have any questions, please point them out .

Two 、 The role of integrated learning

φ(◎ロ◎;)φ How to understand the concept of Yi Zhong ？
If you put Learner See it as a strange fire
that Combination strategy It's the skill of fusing different fire （ Burn to death ）
Integrated learning , It is to make the connection between different fires through the skill , And unleash more powerful forces

In practice, the performance is ： The accuracy of a classifier on a sample can only reach 60%, Through ensemble learning, it is possible for the classifier to achieve 60+%,70% Even higher .

3、 ... and 、 Advantages of integrated learning

① High accuracy ： Since it is the fusion of a variety of different fires ( classifier ), If the power is not increased, it is not chicken ribs ？
② Improve the generalization ability ： Because the training of data set is to disrupt , The training set and test set divided after disruption are different , Therefore, the same or the same kind of learner will get different learners after training on different samples , After different types of learners are trained on different samples, the learners will be even more different . Integrated learning integrates so many distributed learners . Look not to understand ？ Don't worry , Then make it more specific , For example, we can cooperate in examinations , Integrated learning is equivalent to five people （ Everyone is a learner ） Review the same content and take a test paper , Everyone may have a gap in the reception of knowledge .A I won't B try ,B No, change again C try , Five people can't count the next , On the whole, it's more than one person （ Of course, the premise is that everyone is not bad ）？ Therefore, the fault tolerance rate of the integrated learner is often higher , The final integrated learner has better performance .

③ You can take advantage of the features of more learners ： Integration is the integration of multiple learners , You don't have to use five of the same learning devices , It can also be a variety of mixed types , A certain kind of learner may have a good recognition effect on one of the categories , Another kind of learner may have a good ability to distinguish other categories , When combined, they complement each other （ Similar to learning from each other ）. As shown in the figure below （ Watermelon book 172 Page about ）. Good but different is good performance , There are many kinds . The test case can be equivalent to the class we want to distinguish （ For example, distinguish between cat, dog and pig ）

Four 、 Disadvantages of integrated learning

① The cost of time increases ： It's easy to understand , It was supposed to train a learner , Because of integration, it is natural to train multiple , Time is multiplied . It should also be consistent with that sentence ：“ There is no such thing as a free lunch ”, The improvement of accuracy is achieved at the expense of time cost .
② The cost of computing power and complexity increase

5、 ... and 、 Combination strategy （ A brief introduction to ）

After getting a bunch of learners, you need to integrate , And this way of integration we call it a combination strategy . The main combination strategies are as follows ：
① The simple average method
② laws and regulations governing balloting （ Absolute majority voting 、 Relative majority voting 、 Weighted voting ）
③ methods of learning （ Used when there is a large amount of data ）

6、 ... and 、 Common integrated learning methods

①Boosting, Extended algorithm adaboost（ One of the top ten machine learning algorithms can be said ,Adaboost The appearance of makes boosting The algorithm is implemented in the real sense ）
②Bagging【 Integrated learning ：Bagging(bootstrap sampling)】
The origin of the paper
③Random forest（ Random forests ：bagging Extended algorithm ）
④Stacking

The main points of this chapter

① Integrated learning is the integration of multiple “ One （ classifier ）” Synthesis of a “ One ”, Get a stronger learner than the original one
② There are many kinds of learning devices , The effect is good , The integrated learning device combined has better effect .

ps： The practical application of the above integrated learning algorithm will be updated in the future （ It is mainly the combination of integrated learning and deep learning ）

Realization ：python+tensorflow