Playwright tutorial (I) suitable for Xiaobai

One 、playwright brief introduction

playwright It is an automated testing tool released by Microsoft , Code can be generated automatically

I'm learning playwright Can understand before playwright file , Currently only in English ！

1.playwright brief introduction

pip install --upgrade pip
pip install playwright
playwright install

 2. Basic use

import xlwt
#  Import sync_playwright Method 
from playwright.sync_api import sync_playwright
#  Definition run() Method  
def run(playwright):
    #  call playwright Of chromium Create a browser instance , To run the script, you need to first open the browser instance 
    #  If not set to  False, The default is headless mode to start the browser , We can't see any windows 
    browser = playwright.chromium.launch(headless=False)
    #  establish context object ,context Running resources are isolated , It can be understood as a lightweight browser instance 
    context = browser.new_context()
    #  return page object , New page 
    page = context.new_page()
    #  Load Douban webpage 
    page.goto("https://movie.douban.com/")
   

3. Unique features , Automatic generation （ One paragraph is enough ）

playwright codegen -o baidu.py 

-o The following is the file name saved after the code is automatically generated

Two 、 Practice Douban movie TOP10, And generate Excel form

1. Learn about selectors

stay playwright Middle selectors can be used CSS Selectors ,xpath Selectors , The latter can also be applied to other python library

xpath Tutorial can refer to rookies （ Not many things , Some rules , A look at will understand ）

#  Use CSS Select the movie list position with the selector 
    names = page.query_selector_all("#billboard > div.billboard-bd > table > tbody > tr > td > a")
    #  Use xpath Selector get 
    # names = page.query_selector_all("//*[@id='billboard']/div[2]/table/tbody/tr/td/a")

The two selectors are essentially the same ,xpath Selectors can be more intuitive

2. Use the selector in F12 The developer finds the required data location according to

  The range of elements selected by the selector should not be too large （ If it's too big, you can also get , But it's more troublesome , Image acquisition TOP10 This kind of data , Regular , You can select each small label ）, Pictured above a label .,10 individual a label , It's exactly the same as 10 Data .

    #  New list    Used to store movie names and links 
    listName = []
    #  Because the movie list data are <tr><td> style , Traverse each style 
    for name in names :
        #  obtain a The movie name in the node 
        content = name.text_content()
        #  obtain a Movie links in nodes 
        link = name.get_attribute("href")
        #  Append each line of string to the list 
        listName.append((content,link))
        print(content,link)
    print(listName)

Then traverse this 10 Data , Every one of them name It's just one. <a> label , utilize

name.text_content()  Get the text of the label 

name.get_attribute("href") Take it to the label href The value of the property 

Try printing it out , I won't demonstrate

3. Save to Excel in

Consider before saving , establish excel, Table name , Header , Cell size

 #  establish wordbook object 
    workbook = xlwt.Workbook(encoding = 'utf-8')
    #  Create sheet 
    worksheet = workbook.add_sheet(' Watercress movie TOP10')
    #  Set the header of the worksheet ,write（ That's ok , Column , value )
    worksheet.write(0, 0, label=" Watercress movie TOP10")
    worksheet.write(2, 0, label=" name ")
    worksheet.write(2, 1, label=" link ")
    #  Set cell width 
    worksheet.col(0).width = 4000
    worksheet.col(1).width = 10000

The above listName Traverse , Here the enumerate Method , You can traverse the index and data , Control by index excel Rows and columns in are more convenient

 for i,items in enumerate(listName):
        # print(i,items)
        #  Will get items write in excel in     write（ That's ok , Column , value )
        worksheet.write(i+3, 0, items[0])
        worksheet.write(i+3, 1, items[1])
    #      Save the worksheet 
    workbook.save('Excel_test.xls')

Close the browser , page ,

    page.close()
    context.close()
    browser.close()
# call sync_playwright Method , Return to browser context manager , The assignment is p
with sync_playwright() as playwright:
    run(playwright)

3、 ... and 、 summary

Here we use the simulated browser to get , The code is still relatively elementary , The main experience will be the data storage excel The process of , Subsequent updates will be obtained through the interface , Get the data of different pages ...