One line of code accelerates sklearn operations thousands of times

author |  Fevrey

source | Python Big data analysis

*

1、 brief introduction

scikit-learn As a classic machine learning framework , It has developed for more than ten years since its birth , But its computing speed has been widely criticized by users . be familiar with scikit-learn Your friends should know ,scikit-learn Some of the included in are based on joblib The effect of the operation acceleration function of the library is limited , Can't make full use of computing power .

And today I want to introduce you to the knowledge , It can help us without changing the original code , Get tens or even thousands of times scikit-learn The computing efficiency is improved ,let's go！

2、 utilize sklearnex Speed up scikit-learn

In order to speed up the calculation , We only need additional installation sklearnex This extended library , Can help us to have intel On the processor's device , The operation efficiency is greatly improved .

Take a cautious attitude towards tasting fresh food , We can do it alone conda Experiment in virtual environment , All orders are as follows , We installed it by the way jupyterlab As IDE：

conda create -n scikit-learn-intelex-demo python=3.8 -c https://mirrors.sjtug.sjtu.edu.cn/anaconda/pkgs/main -y
conda activate scikit-learn-intelex-demo
pip install scikit-learn scikit-learn-intelex jupyterlab -i https://pypi.douban.com/simple/

After the preparation of the experimental environment , We are jupyter lab Write test code in to see how the acceleration effect , It's easy to use , We just need to import... Into the code scikit-learn Before the relevant functional modules , Just run the following code ：

from sklearnex import patch_sklearn, unpatch_sklearn
patch_sklearn()

After the acceleration mode is successfully turned on, the following information will be printed ：

The other thing to do is to put your original scikit-learn The code can be executed later , I did a simple test on my old savior notebook, which I usually write and develop open source projects .

Take linear regression , On the sample data set of millions of samples and hundreds of features , It only takes... After acceleration is turned on 0.21 Seconds to complete the training set , While using unpatch_sklearn() After the acceleration mode is forced off （ Be careful scikit-learn Relevant modules need to be re imported ）, The training time increases to 11.28 second , Means through sklearnex We got 50 Multiple times of operation speed ！

And according to the official statement , The stronger CPU A higher percentage of performance improvements can be achieved , The following figure shows the official Intel Xeon Platinum 8275CL The performance improvement results after testing a series of algorithms under the processor , It can not only improve the training speed , It can also improve the speed of model reasoning and prediction , In some scenarios, the performance is even improved thousands of times ：

Officials also offer some ipynb Example （https://github.com/intel/scikit-learn-intelex/tree/master/examples/notebooks）, It shows that K-means、DBSCAN、 Random forests 、 Logical regression 、 Ridge return And other common algorithm examples , Interested readers can download and learn by themselves .

---

The above is the whole content of this paper , Welcome to discuss with me in the comments section ~

Looking back

Google AI Be exposed to self Awakening ？

13 individual python Necessary knowledge , Recommended collection ！

Interesting Python Visualization techniques ！

Low code out of half a lifetime , Come back or " cancer "！

 Share 
 Point collection 
 A little bit of praise 
 Click to see