Baidu Encyclopedia data crawling and content classification and recognition

---

Preface

Recently, I am learning the related content of knowledge map , You need to crawl some structured data . Here is how to crawl the data of Baidu Encyclopedia and extract the effective data code .

---

One 、 Analyze the page structure

The page can be divided into 5 Regions , As shown in the following illustration （ polypropylene Page structure of introduction ）.

https://baike.baidu.com/wikitag/taglist?tagId=76613

Two 、 Use steps

1. Import and stock in

The code is as follows （ Example ）：

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import warnings
warnings.filterwarnings('ignore')
import  ssl
ssl._create_default_https_context = ssl._create_unverified_context

2. Read in the data

The code is as follows （ Example ）：

data = pd.read_csv(
    'https://labfile.oss.aliyuncs.com/courses/1283/adult.data.csv')
print(data.head())

It's used here url Data requested by the network .