Trial of the combination of RDS and crawler

This cloud database RDS It's for , In order to understand this RDS Users of , Also in order to do an automatic data storage crawler on the host , Try this today RDS.
Or feeling the stones step by step to cross the river .

Log in to the database RDS

1. First log in from the console
   
2. Enter instance - mysql 8.0 Of ,20g
   
3. Internal and external whitelist settings

First add the intranet to the white list , According to the help beside, I added 0.0.0.0/0 try , This defaults to the intranet host
Internet first click to apply for Internet address , Then I came down after a while .
Both internal and external websites have web addresses . With these two websites , Theoretically , Both internal and external networks can be accessed .

4. Create account , And log in to view

5. Next mysql Try the client , Connected to , Start building tables , I'm still not used to operating on the command line .
   The client under is ok .
   

6. Now local python Try connecting inside , This needs to be installed first pymysql

pip install pymysql

7. The crawler code starts writing , This time, we mainly try to climb down the model information of e-car.com
   url The rule is http://car.bitauto.com/tree_chexing/+type+"_"+id
   however type and id It's in the brand information , I climbed once before .
8. Import module information

import pymysql
import requests as rq
import re
import bs4
import json

9. Database connection , It's still very simple
   Fill in the intranet address , Just fill in the extranet , There is .
   

10. Read the newly filled information from the database , Limited space , Only this and nothing more
    Reptiles mainly use requests, BeautifulSoup Still practicing .

# Get... From the database brand Information is stored in the data dictionary 
dict_car_brand={}
if bl_get_dict_car_brand is False:
    try:
        #  perform SQL sentence 
        cursor = db.cursor()
        sql="SELECT id,type,name,url FROM car_brand"
        cursor.execute(sql)
        #  Get a list of all records 
        results = cursor.fetchall()
        for row in results:
            int_id = row[0]
            str_type = row[1]
            str_name = row[2]
            str_url = row[3]
            #  Add a dictionary 
            dict_car_brand[str(int_id)]={"type":str_type,"name":str_name,"url":str_url}
        bl_get_dict_car_brand=True
    except:
        print ("Error: unable to fetch data")
    finally:
        cursor.close()
#print(dict_car_brand)